369866 - Date sort in "Search Messages" results window or save search/virtual folder does not work correctly

Status: NEW

(Thunderbird :: Search, defect)

Description

[Ted Miller]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1
Build Identifier: 1.5.0.9 20061207

Actions:
1. I search across multiple folders (e.g. when I select the root of a mail account).
2. I click on the "Date" column heading to sort results by date.

Results: 
Desired result: 

Reproducible: Always

Steps to Reproduce:
1. I search across multiple folders (e.g. when I select the root of a mail account).
2. I click on the "Date" column heading to sort results by date.
Actual Results:  
Messages are grouped by folder, then sorted by date.

Expected Results:  
Messages from various folders are intermingled, as forced by Date sort.

I am using IMAP folders.  I believe (but am not 100% sure) it also happens at home, where I run on Linux with local folders.

Comment 1

[Magnus Melin [:mkmelin]]

->NEW on Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.2pre) Gecko/20070209 Thunderbird/2.0pre ID:2007020903

Tested on an imap subfolder, the messages get sorted by date + grouped by subfolder instead.

Comment 2

[Ted Miller]

More information:
To reproduce,
1. I search across multiple folders (e.g. when I select the root of a mail
account).
2. I click on the "Date" column heading to sort results by date.
3. I click on "Search" button again (with previous search still displayed in box
2. I click on the "Date" column heading to sort results by date.

The key seems to be the second search in the same search box, which I often do.  w.g. I search for emails with "Haiti" in the subject.  Get 300 back, so I add a line that restricts the search to ones that Lynn sent me.  Then I want them sorted by date.

If what I have described is correct in causing the bug, it will probably reduce it's severity, as a simple work-around is to redo the search in a new search box.

Comment 3

[ajbaxter]

I too have found this bug.  Clicking on date after a search in multiple folders does not sort by date for all results, but sorts by date for each folder.
I'd consider this a bug.

Comment 4

[Imran]

I have also experienced this problem and it is definitely a bug. In Thunderbird 2.0.0.4, sort by date does nothing. The results appear to be sorted by folder first and date second, but this is actually the default sort. If I sort by another field, say subject, and then sort by date, nothing changes. In addition, switching from ascending to descending does nothing.

I think this problem is particularly important because it makes saved search across folders pretty much useless as a virtual folder implementation.

Comment 5

[ittayd]

I'm suffering from this problem also. 

Comment 7

[Jürgen 'JiHa' Harter]

This seems not to show on current trunk (testet SM2).

See Bug 392546 for details and screenshot.

Comment 8

[Wayne Mery (:wsmwk)]

Has anyone here EVER seen this work in Thunderbird?

Problem goes back at least to bug 276681, much older than this bug.  I don't know why it was found working in bug 276681 comment 2, but I have *never* seen it work and I doubt it ever did. If it did work in the past, and now doesn't then there is a regression. OTOH perhaps sorted by date within folder was the designed/desired behavior for click on date. 

A simple workaround is after search results are displayed, click sort any column other than date, then click date column.


In reply to comment #7)
> This seems not to show on current trunk (testet SM2).
> 
> See Bug 392546 for details and screenshot.

Jürgen  I don't think it is fixed.  I believe what you and Vincent see is when search displays the results and no column has the sort indicator, when you click date it sorts fine.  (Such would be the case probably after a first time install of seamonkey on a PC - which is true of my VISTA machine)  But subsequent searches+sorts fail per previous behavior model. This is the behavior I see with a vista machine and 2007082203 SeaMonkey/2.0a1pre and I believe with further testing you will find it also.  If so, you can dupe bug 392546 to this bug.

Plus, i don't find any bugs or checked in patches, at least recently, that could have affected this issue in a positive way.

Comment 9

[Vincent Deconinck]

(In reply to comment #8)
> Jürgen  I don't think it is fixed [in SM2].  I believe what you and Vincent
> see is when search displays the results and no column has the sort indicator,
> when you click date it sorts fine.

I can't say anything for Thunderbird, but you are perfectly right regarding SM2: I can reproduce it there too.

Let me re-formulate the problem as follows - sorry for the long post :

- Search Messages *always* returns the results in "scan order" (that is, folder after folder), no matter what column has the sort indicator at that time (my tests show that the Subject and Sender columns are equally affected). That gives the first "broken" feeling.

- To make things worse, when a column has the sort indicator before a search is issued, clicking on it again after the search is done only reverts the list (for performance reason I guess). As no actual sort is performed in this case, clicking the header twice leaves the column in the same wrong order you had before (second "broken" feeling). To get the messages list in the correct order, you have to sort on *another* column, then back on the column you want.

- The fact that many people (including me) concluded that it works in certain cases is due to the fact that the problem doesn't show for the first query unless you set the sort indicator beforehand. Search results are returned in scan order, then clicking on the Date column performs the sort correctly. Although, all subsequent searches will be affected until you close the Search Messages window.

- The fact that most people notice the problem on the Date column is probably because (a) Date sort is what most people want and (b) the problem is made more obvious by the fact that the order seems reliable *per location*. Now I think this "partial order" is just a side effect of the scan process : messages in a folder are probably searched in the order they are stored in the folder, which happens to be chronological most of the time

- Funny results can be obtained by pressing the column header while a search is in progress. Results found up to that point will be sorted, then subsequent results will be added at the bottom

To summarize, a sort is only done on-demand when you click on a column header that doesn't have the indicator. Contrary to what one would think, the indicator does not mean that the sort is "permanent".

Low-cost solution : clear the indicator each time the Search button is hit.

Hi-end solution : insert the results in the correct order while performing the search (However, some might object that updating the list while search is in progress makes reading the first results inconvenient because they would move as new results come in).

Don't hesitate to correct any errors of course

Comment 10

[Wayne Mery (:wsmwk)]

I haven't read your entire message, but skimming it I agree. And I like  the "Low-cost solution" - which should be filed as a new bug if it doesn't already exist.  And perhaps clicking on columns should probably not be permitted until the search is complete.

As to the hi-end solution, which some people might like for good reason - I'm thinking I don't want results display suppressed until they can be sorted.  For example my searches tend to be long, and I like to see that the search is in fact working and getting expected results especially for long searches. So the current search behavior is fine for me, except for the click+column issues.

More history ... 

Presumably this was working in SM 1.0 per Mike (who is very reliable) in bug 106343 - done at the same time he closed thunderbird bug 27668 as working in TB 1.0.  BUT ... neither cites a fixing bug.  Mike, does your recollection go back that far? :)

And an even older suite bug ... bug 107280, still open (but presently with a poorly worded summary)

re: bug 392546 because it is seamonkey, for now I don't think it should be duped to this bug

Comment 11

[Vincent Deconinck]

(In reply to comment #10)
> Presumably this was working in SM 1.0 per Mike (who is very reliable) in bug
> 106343 - done at the same time he closed thunderbird bug 27668 as working in
> TB 1.0.  BUT ... neither cites a fixing bug.

I'm only guessing, but maybe he just confirmed that it works on *the first time*. I got bitten too when testing with SM2 : open search, type in a criterion, hit search, sort : WFM.
However, on the second search (or if you set the sort indicator on a column beforehand), the bug shows up.

(BTW, I assume you meant bug 276681, not bug 27668)

Comment 12

[Mike Cowperthwaite]

I see the results that Vincent describes in #11.  It's true that as the search proceeds, it doesn't sort.  And it's also true that if you click a column that's already selected, it doesn't resort; it only reverses the existing order.  (This is true for some situations in the regular thread pane as well, but those typically are always sorted correctly to start with.)  If you select a different column and the return to the column in question, the data will be resorted as desired.

That could well be the same issue reported in those older bugs, but neither of them listed the steps-to-reproduce to get to that point.

Comment 13

[Ken Seehart]

I sometimes get search results that sort the date field with a string sort (e.g. ['1/15/2007', '11/14/2007', '2/14/2007'].  But I can't seem to reproduce this today....

Comment 14

[James Johnson]

Just wanted to chime in on this one. I am experiencing the same bug. When you try to sort your search results by date (first), it sorts incorrectly. However, if you sort by Sender and then sort again by date, it sorts the date correctly. So I can functionally get around this problem by doing this, but it should still be addressed in the next version update.

I'm running Thunderbird ver 2.0.0.22 with Lightning 0.9 calendar addon.

Cheers!
~James J.~

Comment 16

[Wayne Mery (:wsmwk)]

(get thee out of General)

Comment 17

[Wayne Mery (:wsmwk)]

just saw this again. I had basically forgotten about it.

Comment 18

[Lewis Rosenthal]

Ugh... I can't believe that I am *just* now seeing this (Mozilla/5.0 (OS/2; Warp 4.5; rv:8.0.1) Gecko/20111123 Firefox/8.0.1 SeaMonkey/2.5). I would swear that I did not see this under 1.1.18, which I ran for a long, long time before moving to SM2, though I can't recall after running 2.0.14 for such a while before modern builds whether it reared its ugly head.

I am definitely seeing what Vincent described in his comment #9 from 2007, and this was a simple search (only one criterion - "From contains") which included subfolders. I had 227 matches across two subfolders (out of a dozen).

Comment 19

[Lewis Rosenthal]

BTW, the workaround cited in Wayne's comment #17 does *not* work. In fact, once I click on another column and then back on the date column, the sort indicator (default theme) does not even change (up/down; it does change to the desired column heading), and neither does the sort order.

Comment 22

[iii_iii]

I've noticed this bug and when I click the date column or any other column followed by the date column it does not cure it.

Comment 23

[iii_iii]

Attachment: Screenshot_20200101_073842.png

Screenshot of list of news feeds incorrectly sorted. Notice also the current time is 0741 but there is a news item with a time of 1300.

Comment 24

[iii_iii]

I've just realised the dates are wrong: 2020. Ignore.