297251 - [advanced search] Search Messages resets the sort order / sort column. column choices and order should persist

Status: VERIFIED FIXED

(Thunderbird :: Search, defect)

Description

[(not reading, please use [e-mail] instead)]

[advanced search] searching resets the sort order / sort column

to reproduce:

do an advanced search that returns multiple results.  (I did body contains on a
local folder)

the results appear in date ascending.

change the sort (either date descending or click on another column)

click the search button again and we will revert to date ascending.

Comment 1

[(not reading, please use [e-mail] instead)]

I'm using tbird version 1.0+ (20050604) on windows xp

Comment 2

[Wayne Mery (:wsmwk)]

is "ignores" sort column is a better descriptor than "resets"?  sort indicator is always shown on the last column selected, regardless of the results.  

advanced search results are always returned as article physical order in the datasource - which is not unreasonable given that the (partial) results are displayed as the search progresses through the source. This is even more evident when searching multiple folders with folder name column displayed.

Are you suggesting that sort order indicator be removed after each search results? Or, are you suggesting a different display result, requiring perhaps more fundamental changes?

Comment 3

[Wayne Mery (:wsmwk)]

Seth, comment 2 :)

Comment 4

[Pawel Pohl]

Hi, I came up with this bug doing a search for my problem. I have successfully migrated from Outlook at work, moving 2 years worth of emails to Thunderbird specifically because of Outlook's poor search features. My problem is that when I search for something in Thunderbird, all the messages come up starting with 2007 - which in 99% of the cases means I am searching starting from the least relevant results, and I have to wait about 5 minutes to get to 2009, which is ridiculous when searching for an ftp password I was mailed 3 weeks ago... I have a feeling this would not be difficult to change in Thunderbird? Or am I missing some very easy solution?

Thanks!

Comment 5

[Pawel Pohl]

So, what I am actually requesting is that the search order is either reversed or customizable :)

Comment 6

[David :Bienvenu]

(In reply to comment #5)
> So, what I am actually requesting is that the search order is either reversed
> or customizable :)

This bug isn't what you want - you want the search to actually happen in reverse order, looking at newest messages first. This bug is about the order that we display the results, not the order that we calculate them.

for local searches, as opposed to imap, we could probably do this. For IMAP, we're somewhat dependent on what the imap server does.

Comment 7

[Pawel Pohl]

Really fast reply, thank you.

I am talking about a  local search.

Should I file a separate feature request then?

Comment 8

[David :Bienvenu]

yes, thx.

Comment 11

[Mark Banner (:standard8)]

Not blocking 3.1 on this as it doesn't seem a big user issue.

Comment 14

[Wayne Mery (:wsmwk)]

sorry for spam - I failed to examine component of this bug against bug 260289 :(

Comment 16

[u388058]

This does not appear to me to be a dup of 260289, which is specific to the address book.

I think I've got the same problem.  In my case, I think it has to do with secondary sort.  

It didn't used to be this way!  This used to happen in 3.<something> timeframe, but it was fixed somewhere along the way; did not happen, for example, in 9.x (and I don't think in 8.x).  It's now back with either 10.0 or 10.0.1, and it continues in 10.0.2.  

I'm currently running 10.0.2.  I experience this issue in unified folders and in saved search folders.  My mailnews.default_sort_order is 2 and mailnews.default_sort_type is 18 (descending byDate).  On some folders, I have a secondary sort to bring flagged (starred) items to the top (starred by descending date, then everything else by descending date).  

**For those folders with a secondary sort:**

When I performing a quick filter, the order for the results is something I can't quite figure out (doesn't always have a recognizable pattern, but it is sometimes date ascending) -- I would expect it not to change the sort at all from the original folder's display (starred by desc date then desc date).  The secondary sort is usually followed (so I get starred items in random order, then everything else in random order).  

So the results seem to ignore the primary sort and the default sort, but they pick up the secondary sort.  

I would think that the folder has one sort, no matter if quick filter is applied or not, so that if you do not change the sort under quick filter, the original does not change; and if you DO change the sort under quick filter, the original DOES change.  But that's not what I am experiencing.  

Sometimes (but not usually), performing a quick filter changes the sort of the original folder (when I do not change any sorts).  I haven't been able to determine a pattern.

Sometimes (but not always), if I change the sort order while in the middle of a quick filter, the original folder is also changed.  I haven't been able to determine a pattern.  

I haven't (yet?) experienced the problem on a folder with simple sort (which matches the default, desc byDate), so in my case, the issue is with sorting of quick filter results on folder that have a secondary sort.

-- mdeck

Comment 17

[u388058]

p.s. I'm on Win 7 64-bit.

Comment 18

[Wayne Mery (:wsmwk)]

possible to do here what squib did in bug 528044 ?

Comment 23

[alta88]

Attachment: searchCols.patch

1. share threadTree in an inc file for search access to all columns.
2. make user's column selection and sort order persistent for search.
3. remove some dead attributes from threadTree.

this depends on Bug 545906 changes else it won't apply.

Comment 24

[klaus]

can you also add:

1. attachment column
2. message window so that one can see the message highlighted in the search result window?

Then, this woud behave as most other message lists.

Both is very important for business use, if many emails are found for the same sender:

1. helps: which was the email having the contract as attachment?
2. of the ten messages shown with attachment (using my addon) - display message text so that I know which has the contract, which only the quotation or other attachment..

both are daily requirements in business use.

Current workflow needed : click through all the messages and display to find the right one.

many thanks,

Klaus

Comment 25

[alta88]

(In reply to klaus from comment #24)

can you also add:

1. attachment column

The same standard columns in threadpane will now be shown in search.

2. message window so that one can see the message highlighted in the search result window?

No, if you want that then the search should be saved and it will appear as a folder and you can view the message content in 3pane. For single folders, use quick filter in 3pane.

Then, this woud behave as most other message lists.

Both is very important for business use, if many emails are found for the same sender:

1. helps: which was the email having the contract as attachment?
2. of the ten messages shown with attachment (using my addon) - display message text so that I know which has the contract, which only the quotation or other attachment..

both are daily requirements in business use.

Current workflow needed : click through all the messages and display to find the right one.

I looked at your addon and it will no longer be necessary in search dialog. As for searching content within an attachment, that's something very different and there are other bugs requesting that.

My addon, Attachment Count, adds more info to the column (in 3pane threadpane) including single attachment text preview on hover, and may or may not be of any use. https://bitbucket.org/alta8888/attachmentcount/wiki/Home

many thanks,

Klaus

Comment 26

[klaus]

attachment column: very good, also, I won't have to update my addon in the future.

3 pane view: thanks for the workaround with saved search.
Still, that is very impractical. Everytime I search an attachment, contract or other, I would need to make a saved search and then later remove the saved search.

My business partners never change the subject line, so it would be good to be able to look at message text in a 2 pane view (message list and single message display). Such a need is so obvious that I am surprised that it is not already there (may be difficult because search is a specialised folderDisplayWidget?).

If one could see the message text in the standard message display, one could also see whether there are attachments and what is the attachment name. Full search in attachments was never needed by us in 20 years, it was sufficient just to look at attachment names (being file names, they are usually more unique than the message subject lines).

One can see a little bit of message text in the single folder search, but often, to see full message text is needed in a full account (or subfolders..) search.

Comment 27

[alta88]

(In reply to klaus from comment #26)

attachment column: very good, also, I won't have to update my addon in the future.

3 pane view: thanks for the workaround with saved search.
Still, that is very impractical. Everytime I search an attachment, contract or other, I would need to make a saved search and then later remove the saved search.

I don't know what your search rules are, but do you realize you can have a saved search folder that just includes an "attachment status" and "has attachment" rule, in 3pane? Did you know you can use gloda to create a faceted search and open those results in a tab that contains the message list and message content like you propose here?

There are numerous existing ways to get search views. The search dialog view is the ugly child of db views, if you know the codebase.

Anyway, this is now offtopic and not for bugzilla.

Comment 28

[Richard Marti (:Paenglab)]

Comment on attachment 9081980 [details] [diff] [review]
searchCols.patch

Review of attachment 9081980 [details] [diff] [review]:
-----------------------------------------------------------------

::: mail/base/content/threadTree.inc.xul
@@ +29,5 @@
> +                                 fixed="true"
> +                                 cycler="true"
> +                                 currentView="unthreaded"
> +#ifdef SEARCH_WINDOW
> +                                 <!-- The search view does not support threading. -->

This comment inside the element breaks the search window with a malformed error.

Comment 29

[alta88]

Attachment: searchCols.patch

Comment 30

[Richard Marti (:Paenglab)]

Comment on attachment 9082857 [details] [diff] [review]
searchCols.patch

Review of attachment 9082857 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks, looks good and works. r+ with the comment addressed.

::: mail/base/content/SearchDialog.xul
@@ +12,5 @@
>  <?xml-stylesheet href="chrome://messenger/skin/folderMenus.css" type="text/css"?>
>  
>  <!DOCTYPE window [
> +  <!ENTITY % messengerDTD SYSTEM "chrome://messenger/locale/messenger.dtd" >
> +  %messengerDTD;

Now with the messenger.dtd added, the strings for the columns are doubled. You can now remove them from SerchDialog.dtd.

Comment 31

[alta88]

Attachment: searchCols.patch

ah right, removed dupe strings; there's some other bug about it. anyway, if aceman has any substantive comments, i'll do them in a followup. one r+ is enough here.

Comment 32

[:aceman]

Comment on attachment 9082894 [details] [diff] [review]
searchCols.patch

Review of attachment 9082894 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks, this looks good to me. I thought that the column selection is now shared between the main thread pane and the search window, but that is not the case.

::: mail/base/content/SearchDialog.js
@@ -240,5 @@
>  
> -  gFolderDisplay.setColumnStates({
> -    subjectCol: { visible: true },
> -    correspondentCol: { visible: Services.prefs.getBoolPref("mail.threadpane.use_correspondents") },
> -    senderCol: { visible: !Services.prefs.getBoolPref("mail.threadpane.use_correspondents") },

Is this solved in the Search dialog with some other code now? Or is it done by the automatic restoration of columns and this code was only needed when we always initialized the search columns from scratch?

Comment 33

[alta88]

(In reply to :aceman from comment #32)

Comment on attachment 9082894 [details] [diff] [review]
searchCols.patch

Review of attachment 9082894 [details] [diff] [review]:

Thanks, this looks good to me. I thought that the column selection is now
shared between the main thread pane and the search window, but that is not
the case.

No, and it shouldn't be. In fact, it can't be since in threadpane the column selection is folder specific and stored in the .msf db.

::: mail/base/content/SearchDialog.js
@@ -240,5 @@

• gFolderDisplay.setColumnStates({
• subjectCol: { visible: true },
• correspondentCol: { visible: Services.prefs.getBoolPref("mail.threadpane.use_correspondents") },
• senderCol: { visible: !Services.prefs.getBoolPref("mail.threadpane.use_correspondents") },

Is this solved in the Search dialog with some other code now? Or is it done
by the automatic restoration of columns and this code was only needed when
we always initialized the search columns from scratch?

The search dialog column selection and sort is restored based on persisted dom attributes (used to be xulStore.json but now is in xulStore/data.mdb), since there's no backing db, as there is for folders in threadpane.

Btw, someone should do the same dedupe for the addressbook list and search, along with making those colpickers sticky.

Comment 34

[Jorg K (CEST = GMT+2)]

Something we should backport? Looks like an old bug.

Comment 35

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/f6e91a51bd51
Persist search dialog list column choice and sort column/order; share threadTree between 3pane and search. r=aceman

Comment 36

[Richard Marti (:Paenglab)]

Attachment: searchCols-beta-ESR.patch

Bug 545906 also got the ESR approval. And it makes sense to backport. The string changes are only removals, so no problem.

Comment 37

[Jorg K (CEST = GMT+2)]

Comment on attachment 9082944 [details] [diff] [review]
searchCols-beta-ESR.patch

Bug 545906 hasn't been approved yet. It's a bit tricky since it has an M-C part. Should this thing here go through a beta first?

Comment 38

[Jorg K (CEST = GMT+2)]

TB 69 beta 3:
https://hg.mozilla.org/releases/comm-beta/rev/d16312466cccd7771275d32ee45d88474f7f6c83

Comment 39

[Jorg K (CEST = GMT+2)]

Appears to be working in the beta.

Comment 40

[Jorg K (CEST = GMT+2)]

Comment on attachment 9082944 [details] [diff] [review]
searchCols-beta-ESR.patch

Doesn't apply due to the closemenu="none" stuff which isn't on beta, but is on trunk and esr68.

Comment 41

[Jorg K (CEST = GMT+2)]

TB 68.0 ESR:
https://hg.mozilla.org/releases/comm-esr68/rev/38fb021b6113d7876673d8aeaad54c2c31b3a67b

Comment 42

[Jorg K (CEST = GMT+2)]

Also working in TB 68 ESR.

Comment 43

[Dan Pernokis]

Today is 09-Feb-2021 and I'm running the latest TB 78.7.1 (standard user release). The SORT issue has either regressed or still persists -- the sort order still changes for every search within the current session and reverts to a default.

If I sort by something (say, "DATE ^"), then close the search panel, that something becomes the default for the next session. When I open a new search, the first sort will be by this default, and likewise all subsequent sorts within the session -- even if I re-sort by something else (say, "DATE v"). If I close & reopen, the most recent sort criteria is the new default for the next session, and all subsequent searches in the new session will initially sort by this default.

So then...
If I am in the search panel already, any SORT will work. But any re-SEARCH will revert to the most recent DEFAULT, as set in the above examples.

I expect and want the SORT -- in the current SEARCH session -- to persist as I most recently set it in this (current) session, such that a re-search or new search will come up sorted in that order. Most likely, I'm trying to find and isolate something, and having to re-change the SORT parameter or re-flip the sort order is a major distraction.