26131 - Sorting threads in 3-pane is very slow

Status: VERIFIED FIXED

(MailNews Core :: Backend, defect, P1)

Description

[sol]

Sorting threads in 3-pane is very slow.

Build ID: 2000-01-31-11

Steps to reproduce:

1. Launch Mail
2. Select Inbox (mine contains about 2K messages)
3. Sort by date
4. Sort by sender

Expected result: Sort operation is similar in speed to 4.7
Actual result: Sort operation is much, much slower (I would guess 5 to 10 times 
slower)
Note: The speed of the sort seems loosely correlated with the size of the folder 
being sorted

Comment 1

[scottputterman]

I'm marking beta1.  According to the latest perf numbers, it takes about 8.2 
seconds on a Windows target machine vs 1.51 in 4.7.

I'm not sure that we should actually hold beta for this, but I think that if I 
find stuff that speeds this up, I should be allowed to check in. So I wouldn't 
mind having the PDT+ team's approval to do so.

Comment 2

[scottputterman]

putting UNKNOWN in the status whiteboard.  I could probably keep working on perf 
bugs until I'm told to stop.

Comment 3

[Phil Peterson]

In light of unknown difficulty, PDT- for beta1

Comment 4

[scottputterman]

marking M20 since this is probably going to be ongoing throughout the project.

Comment 5

[lchiang]

Suresh - can you post the latest numbers / timings here?  Thanks.

Comment 6

[Suresh]

Update on recent builds: Sorting by subject and reversing the sorting order 
takes ~8 seconds on the target machine. 

Comment 7

[scottputterman]

moving to m18 and nominating nsbeta3 since this is something we said we are 
going to focus on.

Comment 8

[nhottanscp]

Attachment: patch to improve sort performance, r=rjc

Comment 9

[scottputterman]

Mail triage marking [nsbeta3+]

Comment 10

[scottputterman]

Naoki, were you able to get this in?

Comment 11

[nhottanscp]

Okay, I will check in the patch.

Comment 12

[Robert John Churchill]

The idea struck for a small speed improvement: inverting a current sort doesn't 
require an actual re-sort... the column in question can just be inverted.  For 
example, if a "Name" column is currently sorted in ascending order (1..n), and 
the user wants to sort descendingly on "Name" (from n..1), instead of resorting 
(touching RDF), the nodes in the content model can just be inversely shuffled.

I worked up a diff and added some quick debugging time info (on Mac). I didn't 
see any speed improvement on a Mac for small folders (say around 100 items). 
Would anyone care to test this on a big mail folder (say 5,000 to 10,000 msgs) 
and get a timing to see if the code change is worthwhile?

I'll attach the diff.

Comment 13

[Robert John Churchill]

Attachment: Diff of experimental sort change; if inverting a sort on a column which is already sorted, just invert content model nodes

Comment 14

[scottputterman]

For this case this patch is really good.  I tried a 5000 message local folder 
and had the following results:

Sort by Sender: First = 3.815s Reversed = 1.713s
Sort by Date:  First = 2.704s Reversed = 1.742s

And a 8839 message newsgroup:

Sort by Sender: First = 8.022s Reversed = 3.344s
Sort by Date: First = 4.937s Reversed = 3.495s

Comment 15

[Robert John Churchill]

Thanks, Scott.  Guess I'll check the changes in.  nhotta, I'll check in your fix 

as well.

Comment 16

[lchiang]

rjc - just wondering, did your patch ever get checked in?

second pass: - per mail triage

Comment 17

[Robert John Churchill]

lchiang:  yes.

Comment 18

[scottputterman]

nominating for mail1

Comment 19

[[e-mail] (gone - use [e-mail] instead)]

I've created bug 63759 which is a performance tracker bug for mail/news.

Comment 20

[scottputterman]

marking nsbeta1+ and moving to mozilla0.8 though it might get moved out as more
research is done.

reassigning to sspitzer

Comment 21

[Keyser Sose]

*** Bug 65468 has been marked as a duplicate of this bug. ***

Comment 22

[scottputterman]

moving to mozilla0.9

Comment 23

[James Green]

OS/All, plus keywords.

Comment 24

[(not reading, please use [e-mail] instead)]

fixed.

hats off to hyatt, mscott and bienvenu.

there is some room for improvement (in author sort and subject sort) but that
will be covered in another bug.

Comment 25

[[e-mail] (gone - use [e-mail] instead)]

http://www.mozilla.org/mailnews/win_performance_results.html

This is fixed in my latest testings.  There is still room for improvement like
Seth mentions, but in general, it's been improved by leaps and bounds.  VERIFIED
FIXED.  That link cycles through Mac and Linux results, as well.