522908 - [faceted search] Search results don't update automatically to include newly arrived messages (both faceted and "show all as list"). Need refresh capability

Status: NEW

(Thunderbird :: Search, enhancement)

Description

[Thomas D. (:thomas8)]

STR

1 do "search all messages" for "bug"
2 click "open all as list"
2 receive 1 bugmail from bmo (see the biff popping up)
3 look at your list results (search results threadpane)
4 look at your faceted results (previous tab)
Don't close tabs, we'll need them lateron

Actual result
3 new messages is not included in list results (389 msgs)
4 new message is not included in faceted results (389 msgs)
 
Expected result
- new message is indexed on arrival (which works, I think)
- new message is automatically updated/included/added in my current search results

I actually hope I have done something wrong so that this turns out some bad dream or a user error. Otherwise, I just can't believe that we're not doing what even age-old quicksearch filters do without problem - automatically update search results on arrival of new mails.

Another problem after arrival:
Repeat step 1: do "search all messages" for "bug"
Repeat step 4: look at your /new/ faceted results (previous tab)
-> be confused because it's still 389 msgs (as above, before arrival of new mail)
5 now use date picker for today's date
a) on your old faceted results
b) on your new faceted results
6 Press "show more" on /new/ faceted results until you see all of today's messages matching bug (and see that we need "show all"), then look at the count on top of faceted previews: 40 of 389(!)
7 Press "show more" on /old/ faceted results until you see all of today's messages matching bug, then look at the count: 39 of 389.

That's impossible. If new result actually includes the new msg (which it does, as you can see on the day's list count change, and you also see the new msg in preview results), total count can't be the same as without the new msg.

And sorting of faceted results doesn't work (Bug 519818), so you have a hard time finding your message anyway. Sorry, but I truly love quick filters.

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

Yes, quick-search filters still have a lot going for them.

This bug is really a few separate problems:

1) The faceted search UI code does not handle changes to the set of messages matching the query.  It probably would want to do the webbish thing of "There are N new messages that match this query!" rather than updating things in real-time.  This avoids a lot of potential annoyance involving things in the UI changing without user interaction as well as avoiding the situation where a query result that was limited shows new results that would not actually have been returned because of the limit in scoring heuristics in use.

2) Gloda queries don't support in-memory testing of full-text constraints.  This was an intentional punt because FTS3 is potentially doing a lot of stuff under the hood that we can't currently get at.  (Porter stemming being the main thing, but there are also complications involving multi-column search and synthetic columns that only exist for fulltext-search purposes.)  Most of these can be overcome, or at least overcome enough to handle most common use-cases.

Comment 2

[Wayne Mery (:wsmwk)]

accidentally set blocking?
but leaving as is to see what shakes out b/c #1 would be nice

Comment 3

[David :Bienvenu]

I don't think this blocks 3.1

Comment 5

[Chris Bryant]

I added a feature request for this as well and just marked it as dup of this. I also recommended it be implemented as Andrew suggests in comment 1. Here is what I had written:

When searching with Thunderbird 3's new search (which is really amazing btw,
thanks!) the search can't be refreshed/updated. It would be ideal if there was
a refresh button on the search tab or if Thunderbird displayed some alert near
the top of the tab that said something like "There are changes to your search,
would you like to update the search now?: Yes/No"

This would save you from having to close the search and then open a new one in
order to see changes to the search results (such as changing message tags.)

Also, this is different but related to another enhancement request which only covers a refresh for a saved search but it's worth mentioning: 365393
(https://bugzilla.mozilla.org/show_bug.cgi?id=365393)

Comment 6

[Thomas D. (:thomas8)]

This will be a major problem for tb2 users in transition223, because they only know quickfilters where the results update automatically without intervention, so they'll expect the same for tb3. It's really bad we force people to do a *new* search just to update the results of an existing search.

There are several possible solutions to this problem, e.g.
1) implement an "update/refresh search results" button (for manual refresh, works even if TB doesn't realize there's new messages that match)
2) show link "there are N new messages that match" (per Andrew's comment 1, 1). I'd agree that's better for the graphical search results
3) automatically update the results list (I'd maintain that this would be better for any "list-style" results, like in "Open as list" - can't see any reason why these shouldn't update automatically, as they currently do in tb2 and 3 with quicksearch.

Comment 7

[Shari Smith]

This bug & bug 365393 seem to be about the same thing.  [actually no, 365393 is about saved search]
I move messages to certain folders, based on the search. What I'd love to see implemented is. Once the search is "stale" a refresh button shows up. This would allow me to refresh when I'm ready to do so, as well as let me know things have changed. I also think an F5 should produce the same results.

I'd also add that the "open as list" should look more like a button. I had found it, and then the next time I went to look for it, it took me a while, as it's in an "information" row and doesn't really stand out like a button.

Comment 8

[Ludovic Hirlimann [:Usul]]

Can't block if the backend part is not present.

Comment 9

[Wayne Mery (:wsmwk)]

workaround is reissue search. so sev=minor