597688 - selecting a newsgroup hangs overnight; so does re-sorting its messages

Status: RESOLVED INCOMPLETE

(Thunderbird :: Folder and Message Lists, defect)

Description

[Tom Roberts]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6
Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2

After reading 4 newsgroups successfully, selecting the fifth hung overnight. I have had to do ForceQuit on Thunderbird several dozen times in the few days since upgrading to 3.1.2, on two different Macs, all due to this one newsgroup: sci.physics.relativity. It is hosted on text.giganews.com, on which it has >500,000 messages. I told Thunderbird to download 5000 headers and mark others read. Sometimes I could read its messages, and at one crash only 4 unread messages remained (displaying only unread messages), it hung >30 minutes after clicking on the Subject header (to sort messages).

This time it hung overnight when I selected the newsgroup.

I have been completely unable to read this newsgroup more than once between crashes.

Release 2.X was fine reading this newsgroup. I have used Thunderbird for both email and newsgroups (including this one) for many years.

Reproducible: Always

Steps to Reproduce:
(happened several times):
1.Subscribe to sci.physics.relativity
2.tell it to download 5000 headers, mark others as read (messages are displayed)
3.it comes up sorted by date; manually mark all but the last dozen as read
4.select some other newsgroup, then select sci.physics.relativity again (this will remove the 4988 read messages from the list of messages)
5.click on the Subject header (to sort by subject)

ForceQuit on Thunderbird, and re-run Thunderbird.

Alternate Steps (happening to me now):
1. select newsgroup sci.physics.relativity
Actual Results:  
Hangs > 30 minutes (spinning beachball)
Thunderbird uses ~100% of 1 CPU, 16 threads, 87.61 MB memory, no network activity, no or very low disk activity.


Expected Results:  
sort the messages and display them

select the newsgroup, sort its messages, and display them

It is of course possible this is a single problem -- selecting the newsgroup requires it to sort the messages.

Same problem on TWO machines, both running Mac OS X 10.5.8. One is a MacBook (CPU is core 2 duo, 2 GB memory), the other is a Mac Pro (dual 2-core Xeons, 6 GB memory).

Comment 1

[Tom Roberts]

I just noticed that while Thunderbird updated itself a few days ago, it got 3.1.2, but 3.1.4 is out. I just installed 3.1.4 on my MacBook.

It read email and 4 other newsgroups OK, but it immediately crashed when I selected sci.physics.relativity. I restarted it, unsubscribed, exited, restarted, subscribed, downloaded 5000 headers marking others as read, and could read messages, mark them read, sort by clicking on column headings. I'm displaying only unread messages, and marked all read.

I then read my INBOX, and then clicked on sci.physics.relativity -- hang >5 minutes; ForceQuit.

So 3.1.4 looks like it helped a bit, but still hangs and crashes on this newsgroup.

I'm going back to 2.X until this is resolved.

Comment 2

[WADA:World Anti-bad-Duping Agency]

Sounds similar problem to bug 541001, as huge newsgroup.
Note: bug 541001 is closed as dup of bug 185634.

Comment 3

[Tom Roberts]

I don't think this is really the same as bug 185634 -- that one dates back to 2002, while this problem appeared between Thunderbird 2.0.0.14 and 3.1.2. Thunderbird 2.0.0.14 is quite usable on this newsgroup (and uses only 80 MB of "real memory" on my Mac); 3.1.2 and 3.1.4 are not usable at all on it.

As I remarked in 185634, from the description above it looks to me like this is a new bug, probably related to sorting messages in large newsgroups. I suppose it could be something I would not have noticed, such as lots of short connections to the server, mostly waiting on the network and using <5 kb/s (my server connection sometimes delays several seconds).

Comment 4

[Wayne Mery (:wsmwk)]

(In reply to comment #3)
> I don't think this is really the same as bug 185634 -- that one dates back to
> 2002, while this problem appeared between Thunderbird 2.0.0.14 and 3.1.2.
> Thunderbird 2.0.0.14 is quite usable on this newsgroup (and uses only 80 MB of
> "real memory" on my Mac); 3.1.2 and 3.1.4 are not usable at all on it.

Tom, the amount of memory used in version 3.1.4 is the same when in the newsgroup? (per comment 0)

In bug 185634 you write, "In the newsgroup preferences there is an option to "delete message older than 30 days" -- can't that be used to reduce the size of the DB? Indeed it looks like 2.0.0.14 did just that (when I discovered it a few weeks ago before pgrading, its speed went up considerably)."  Is this not working for you in version 3?

Does "repair folder" function help?

Does hang occur if you sort on any column *other* than subject?

Comment 5

[Wayne Mery (:wsmwk)]

Tom...
> Tom, the amount of memory used in version 3.1.4 is the same when in the
> newsgroup? (per comment 0)
> 
> In bug 185634 you write, "In the newsgroup preferences there is an option to
> "delete message older than 30 days" -- can't that be used to reduce the size of
> the DB? Indeed it looks like 2.0.0.14 did just that (when I discovered it a few
> weeks ago before pgrading, its speed went up considerably)."  Is this not
> working for you in version 3?
> 
> Does "repair folder" function help?
> 
> Does hang occur if you sort on any column *other* than subject?

Joshua, do we need a stacktrace for this?

Comment 6

[Joshua Cranmer [:jcranmer]]

(In reply to comment #5)
> Joshua, do we need a stacktrace for this?

Stacktraces are always helpful.

Comment 7

[Ludovic Hirlimann [:Usul]]

Tom can you follow the instructions at https://developer.mozilla.org/En/How_to_get_a_stacktrace_for_a_bug_report and provide us with a stack trace ? (replace firefox with thunderbird in those instructions)

Comment 8

[Joshua Cranmer [:jcranmer]]

RESOLVED INCOMPLETE due to lack of response to previous question. If you feel this change was made in error, please respond to this bug with your reasons why.