234318 - Slow sorting news

Status: RESOLVED WORKSFORME

(MailNews Core :: Networking: NNTP, defect)

Description

[Michael Leytus]

User-Agent:       
Build Identifier: http://ftp.mozilla.org/pub/mozilla.org/mozilla/releases/mozilla1.6/mozilla-win32-1.6-installer.exe

I have Pentium III 1Ghz, 512Mb RAM, with Windows 2000 Pro

I open a big newsgroup that contains about 16,000 messages. It is sorted by 
threads. Then I click a "Sender" button to sort them by sender. Sorting takes 
33 seconds! Then I click "Threads" button to sort messages by threads again. 
And it takes thirty three seconds again!!

However when I open this newsgroup on the similar PC by Netscape 7.1, sorting 
by sender takes just two seconds, and sorting by threads takes less than one 
second!

Reproducible: Always
Steps to Reproduce:
1. Install Mozilla 1.6 for Win32
2. Open some newsgroup server. For example, news://ddt.demos.su
3. Open a really big newsgroup (10,000 messages or more). For example, 
fido7.russian.z1
4. Sort it by sender.
5. Sort it by threads.
6. Close Mozilla, because the second sorting goes faster.
7. Run Mozilla again and sort one more time to make sure the problem still 
occurs.


Expected Results:  
Can you just take the sorting algorythm from Netscape 7.1?

Comment 1

[Jo Hermans]

(In reply to comment #0)
> Can you just take the sorting algorithm from Netscape 7.1?

Reporter, do you claim that Mozilla is slower than Netscape 7.1 ? We're using
the same code.

Comment 2

[Dmitry Nikiforov]

In Mozilla 1.6 sorting a message list really takes a VERY LONG time, even on
Athlon-64 3000+ (approximately 5-6 seconds to sort a list of ~8000 messages). On
the same box, Netscape 7.1 takes less than a second to sort exactly the same
list of messages.

Comment 3

[Dmitry Nikiforov]

Just an addition to my previous comment: it is true for both, Windows and Linux
versions of Mozilla.

Comment 4

[Michael Leytus]

(In reply to comment #1)
> Reporter, do you claim that Mozilla is slower than Netscape 7.1 ? We're using
> the same code.

I just described what I had seen.

I know that you are using the same code, so I am also wondering why this 
occurs. I can guess that something was changed in sorting algoritm in Mozilla 
1.5 and 1.6, because Netscape 7.1 is based on 1.4, and I did not check it in 
1.4.

Comment 5

[Cyril Novikov]

> Reporter, do you claim that Mozilla is slower than Netscape 7.1 ? We're using
> the same code.

Something definitely changed. I have Mozilla 1.3 on one machine (P4 2.8GHz) and
1.6 on another (P4 2.66GHz). It takes Mozilla 1.6 about 10 seconds to sort a
newsgroup which only takes a fraction of a second to sort using Mozilla 1.3. I
use "keep all messages" option and the same news server.

Comment 6

[Jo Hermans]

This is the list of changes to nsMsgDBView.cpp, but I don't see anything obvious
for a big performance regression. It might be at a different location.

http://bonsai.mozilla.org/cvslog.cgi?file=mozilla/mailnews/base/src/nsMsgDBView.cpp

Comment 7

[Anton Solovyev]

I have not tried Netscape for a while, but Mozilla is painfully slow to sort
news messages when there are several thousand.

Also, it takes 3-5 seconds to advance to next unread message in such newsgroup
when threaded view is in place.

This is extremely disabling for the news client.

Comment 8

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 9

[Gervase Markham [:gerv]]

This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.

Comment 10

[Michael Leytus]

It looks like I found out why this bug takes place. When you sort newsgroup by
sender, and then back by threads, it opens ALL threads in the newsgroup. And if
there is few hundred threads, it takes a hell of a lot of time. And Netscape 7.1
(Mozilla 1.4) probably does not do it.

So you can fix it if you turn off automatic opening of all threads. Or maybe
there is an option in config that can do it? Thanks.

Comment 11

[Wayne Mery (:wsmwk)]

Michael Leytus in comment #10:
> It looks like I found out why this bug takes place. When you sort newsgroup by
> sender, and then back by threads, it opens ALL threads in the newsgroup.

Michael, what do you mean by "open"?  
Does this happen with a trunk build?
  http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk/
or 
  http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-trunk/


Cyril Novikov  in comment #5
> Something definitely changed. I have Mozilla 1.3 on one machine (P4 2.8GHz) and
> 1.6 on another (P4 2.66GHz). It takes Mozilla 1.6 about 10 seconds to sort a
> newsgroup which only takes a fraction of a second to sort using Mozilla 1.3. I
> use "keep all messages" option and the same news server.

This is going WAY back, but can some confirm and narrow this 'regression' if indeedit happened between v1.3 and v1.6?

Comment 12

[Cyril Novikov]

> This is going WAY back, but can some confirm and narrow this 'regression' if
> indeedit happened between v1.3 and v1.6?

Responding to your email asking if this is still true.

I can confirm this is still a problem in 1.7.13. For example, it still takes about 20 seconds to switch between threaded and unthreaded view in a newsgroup of about 25000 messages. This was a long ago now and my memory of good times starts to fade, but I'm sure it took much less than that in Mozilla 1.3, a few seconds at most.

Comment 13

[Michael Leytus]

> Michael, what do you mean by "open"?  

I mean "expand".

Comment 14

[(not reading, please use [e-mail] instead)]

sorry for the spam.  making bugzilla reflect reality as I'm not working on these bugs.  filter on FOOBARCHEESE to remove these in bulk.

Comment 15

[Joshua Cranmer [:jcranmer]]

Sorting news.mozilla.org's n.p.m.mailnews takes no more than two seconds, when switching from unthreaded to threaded, even including threading by subject as well.

Either dupe to bug 226730 or mark this as INVA/WFM. Bienvenu, your thoughts?

Comment 16

[David :Bienvenu]

I think that showing threads expanded *is* quite a bit slower than showing them collapsed, especially if some of the threads are large (which is different from threading messages by the same subject), and that could account for the performance problems. The sorting algorithm itself essentially hasn't changed since netscape 4.

You have to distinguish between threading new messages, i.e., finding the right thread object to put them in, in the db, which was especially slow when threading only by subject, within large threads, and switching to threaded mode, which only reads the threading information from the db which we calculated when the header first arrived. 

Comment 17

[Cyril Novikov]

I just want to confirm once again that switching between threaded and unthreaded views is painfully slow. With a high-traffic newsgroup like alt.russian.z1, the wait is upwards of 20 seconds on my P4 2.66GHz. If you want to verify this with alt.russian.z1, make sure "Don't delete any messages" option is set in your account settings. I've heard from Outlook users on the same newsgroup that Outlook does it an order of magnitude faster. This is a real inconvenience, because switching the views helps to manage the high traffic. I can cope with it by setting "delete messages older than 30 days" -- switch time becomes around 7 seconds (still could be better), but Outlook users don't even have to do that.

Comment 18

[Joshua Cranmer [:jcranmer]]

Marking this as confirmed, but asking for qawanted, as I think other factors are involved.

Comment 19

[Wayne Mery (:wsmwk)]

Cyril, Michael, do you still see this?

Comment 20

[Cyril Novikov]

I guess not. Tried on a 10000 message newsgroup with TB 11.0, the unthreaded->threaded switch takes much less than a second.

Comment 21

[Phoenix]

Resolved per whiteboard and Comment 20