517467 - Group by/Sort by date is broken

Status: RESOLVED FIXED

(Thunderbird :: Folder and Message Lists, defect)

Description

[Ludovic Hirlimann [:Usul]]

1)Make sure to first sort your mail in columns by Date. If you don't have that column displayed, then you will need to click the widget to add that column and then sort that column by date.
2)Click on View | Grouped by Sort

It doesn't display anything. 

console says :

Error: uncaught exception: [Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIMsgDBView.open]"  nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)"  location: "JS frame :: file:///Users/dul/Thunderbird.app/Contents/MacOS/modules/dbViewWrapper.js :: DBViewWrapper__createView :: line 1083"  data: no]

xref bug 511663

Comment 1

[Ludovic Hirlimann [:Usul]]

deleting the local store rdf filr doesn't solve this issue.

Comment 2

[Ludovic Hirlimann [:Usul]]

(In reply to comment #1)
> deleting the local store rdf filr doesn't solve this issue.

Maybe it does - but not immediately I had to rebuild the index and change folder view to get a visible inbox

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

The folder settings are stored in the .msf, not localstore.rdf.  We certainly have seen bugs before where it is possible to get the folder settings into a state that the actual underlying view gets angry about... but I thought we fixed those.

Since it is completely legal to group by date, the most likely explanations are: - that somehow the code thought it should be using a different sort
- that there were other view settings in effect that confused things
- something somewhat unrelated threw an exception and broke stuff leading to the observed problem.

If you can reproduce and provide precise STRs, that would be ideal.

Comment 4

[Ludovic Hirlimann [:Usul]]

1) rm -rf ~/Library/Thunderbird
2) launch thunderbird - create a new account for a gmail email
3) click on the date column so the triangle get upside/ down (ie the flat part on top of the triangle).
4) View -> Sort by -> Group By sort

I can reproduce at will. (I can give you credentials to acces that mailbox if needed asuth)

Comment 7

[ovidiu]

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.4pre) Gecko/20090915 Thunderbird/3.0b4
Gmail imap inbox

confirming here also, had to rebuild index

but I don't consistently go into a blanc tree, I get cute things before, a list of several "this week" and "old mail" lines with no effect or even something like the real thing but refreshing list as I hover the tree

Like this STR:
-sort by date
-group by sort, and It seams to work (short list of msg)
-change date ascending/descending
result:
nothing changes
does change as I hover the tree to a chaotically organized grouping not working

Error: uncaught exception: [Exception... "Component returned failure code: 0x80550008 [nsITreeView.toggleOpenState]"  nsresult: "0x80550008 (<unknown>)"  location: "JS frame :: chrome://global/content/bindings/tree.xml :: changeOpenState :: line 228"  data: no]

should it be bit of a different bug?


usually I get this

Error: uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsITreeView.toggleOpenState]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: chrome://global/content/bindings/tree.xml :: changeOpenState :: line 228"  data: no]


and on the completely blanc tree I get:

Error: uncaught exception: [Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIMsgDBView.open]"  nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)"  location: "JS frame :: file:///C:/Program%20Files/Mozilla%20Thunderbird%203.0%20Beta%204/modules/dbViewWrapper.js :: DBViewWrapper__createView :: line 1083"  data: no]

 +on away and coming back to respective folder:
Error: sortType is undefined
Source File: file:///C:/Program%20Files/Mozilla%20Thunderbird%203.0%20Beta%204/modules/dbViewWrapper.js
Line: 1534

Comment 8

[Henry N.]

(In reply to comment #4)
> 1) rm -rf ~/Library/Thunderbird
> 2) launch thunderbird - create a new account for a gmail email
> 3) click on the date column so the triangle get upside/ down (ie the flat part
> on top of the triangle).
> 4) View -> Sort by -> Group By sort

Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.9.1.4pre) Gecko/20090915 Thunderbird/3.0b4

Error: uncaught exception: [Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIMsgDBView.open]"  nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)"  location: "JS frame :: file:///usr/local/thunderbird-20090915-30b4/modules/dbViewWrapper.js :: DBViewWrapper__createView :: line 1083"  data: no]

I can confirm the same result under Linux. Have also used a fresh TB account with ~13000 mails in inbox of IMAP. The step 3 is necessary. The flat part of triangle is on top per default, but it crashed only if you changed this.

Comment 10

[Andrew Sutherland [:asuth] (he/him)]

Based on brief inspection, I expect the problem is the way we try and approximate support for secondary sort (or more) by constructing the view with the least specific sort and then working our way backwards.  DBViewWrapper._ensureValidSort guards against the primary sort being something that group-by-sort can't support, but does not do anything about secondary (or more).

The absolute simplest solution is likely to change _ensureValidSort to perform a filtering pass on all of the things in the sort stack.

The most correct solution is probably to modify nsIMsgDBView to have a better sort interface, fix up the C++ view implementations and have the DBViewWrapper stop trying to be clever.

The right solution is probably the simplest solution with some additional tests.

Comment 11

[Andrew Sutherland [:asuth] (he/him)]

Attachment: v1 make _ensureValidSort check all sorts and have correct fallover sort, make magicSort use _ensureValidSort's result, add unit tests. patch on top of gloda-delete.

It turns out things are slightly worse than I thought.  _ensureValidSort's results were not affecting magicSort and its updated _sort value was not correctly nested.

This patch adds unit tests that should cover the cases generally related to this situation.

The patch is on top of my gloda-delete test refactoring which is not yet up for real review, so not yet asking for review.

Comment 12

[[:Aureliano Buendía]]

It happens also to me on Windows XP when indexing is in progress.

Changed OS related.

Comment 13

[Ondra Kotecký]

+1 on WinXP. The bug was seen with both -- an IMAP and a POP3 account.

Comment 15

[Martin]

Did some additional study on this after reporting my bug (duplicate)
Here are my results, hope this helps:

Scenario 1
Setup a new profile in TB3
Exit TB3
Copy old inbox (or any other mail file) into new profile (no msf)
Open TB3
TB3 re-indexes inbox
Select inbox > compact
Select View > Sort by > Group by Sort
TB displays messages as expected.
Afterward you can select any sort and then "Group by Sort" and everything works just fine.

Scenario 2
Setup a new profile in TB3
Exit TB3
Copy old inbox (or any other mail file) into new profile
Open TB3
TB3 re-indexes inbox
Select inbox > compact
<b> >> Sort messages by a field << </b>
Select View > Sort by > Group by Sort
All messages disappear

Tried this on all folders and it is repeatable every time!
Looks like if TB writes a file attribute/property (somewhere)(msf file?) that there is a “Group by Sort” then it uses that attribute for further operations and everything works OK.  If that “attribute” (property) is not written before applying a “Sort by” then the display is hosed and messages disappear, even if one can find and display them using search.

Comment 17

[ghuttemann]

The "Scenario 1" in comment #15 worked well for me. Thanks Martin.

Comment 18

[David :Bienvenu]

Comment on attachment 402387 [details] [diff] [review]
v1 make _ensureValidSort check all sorts and have correct fallover sort, make magicSort use _ensureValidSort's result, add unit tests.  patch on top of gloda-delete.

I've applied this patch on top of gloda delete and I'll run with it for a bit.

Comment 19

[Anthony Rosati]

did the following to get mine to come back;

under ~/thunderbird/*.default/
rm localstore.rdf

and restart thunderbird ... folders and messages are back

Comment 20

[Anthony Rosati]

sorry 

~/.thunderbird/*.default/
rm localstore.rdf

Comment 21

[Anthony Rosati]

ignore my replies ... i'm back to messages not showing - the smart folder
didn't solve the issue. sorry

Comment 22

[David :Bienvenu]

Comment on attachment 402387 [details] [diff] [review]
v1 make _ensureValidSort check all sorts and have correct fallover sort, make magicSort use _ensureValidSort's result, add unit tests.  patch on top of gloda-delete.

this fixes Ludovic's reproducible case, and the unit tests pass as well.

Comment 23

[Andrew Sutherland [:asuth] (he/him)]

pushed to trunk:
http://hg.mozilla.org/comm-central/rev/5bc61d53d1ab

pushed to comm-1.9.1:
http://hg.mozilla.org/releases/comm-1.9.1/rev/41ea36609e28

Comment 24

[Thomas D. (:thomas8)]

This should be checked if still occuring after fix for bug 517467 will land today

Comment 25

[Thomas D. (:thomas8)]

(In reply to comment #24)
> This should be checked if still occuring after fix for bug 517467 will land
> today

*lol* sorry, I dropped that comment on the wrong bug, was meant for Bug 378576 -  Problem with Messages sorted and grouped by date when passing midnight