Closed Bug 442674 Opened 16 years ago Closed 15 years ago

Unstable reading large imaps mail boxes on server - hang with very high CPU utilization and constantly increasing memory utilization

Categories

(MailNews Core :: Networking: IMAP, defect)

Product:
MailNews Core
Component:
Networking: IMAP
Version:
1.9.1 Branch
Platform:
x86_64
macOS
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: Jon.Kibler, Unassigned)

References

Details

(Keywords: hang, perf, Whiteboard: closeme 2010-02-11)

Attachments

(2 obsolete files) 

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_3; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.1 Safari/525.20
Build Identifier: version 3.0a2pre (2008063003)

When I have a large imaps mail boxes on server, Thunderbird will often hang with very high CPU utilization and constantly increasing memory utilization. Thus continues for hours until memory is exhausted. It seems to occur most frequently when clicking on a new mail that is older in the mail box than the email currently being read.

This also happens with news groups, especially when you choose to mark all as read.

Engimail and Lightning seem to make it more unstable.





Reproducible: Sometimes

Steps to Reproduce:
1.Use IMAPS on mail server. 
2.Have a large number of email (5k+)
3.Read the most recent unread email
4.Choose an email several previous (older)
5.About 10% to 20% of time, Thunderbird will hang.



I have Talkback installed, but when I do a force quit, it does not pop up. Instead, crash reports get sent to Apple.
Jon, does this fail still with latest nightly?
Severity: normal → critical
Keywords: hang
I have been using the latest for several days now. If anything, it is more unstable than before.

Running top, what is happening is that Thunderbird starts running some process that chews up about 115% cpu, creates a resident working set of 2.5G on a 2G RAM MAC, sucks physical memory down to < 10M, and then either continues to process for anywhere from 30 seconds to over an hour and completes successfully, or crashes with a "Bus Error" as follows:

Sep 27 19:22:31 Macintosh [0x0-0x27027].org.mozilla.thunderbird[315]: thunderbird-bin(315,0xa0075fa0) malloc: *** mmap(size=6234112) failed (error code=12)
Sep 27 19:22:31 Macintosh [0x0-0x27027].org.mozilla.thunderbird[315]: *** error: can't allocate region
Sep 27 19:22:31 Macintosh [0x0-0x27027].org.mozilla.thunderbird[315]: *** set a breakpoint in malloc_error_break to debug
Sep 27 19:22:31 Macintosh thunderbird-bin[315]: thunderbird-bin(315,0xa0075fa0) malloc: *** mmap(size=6234112) failed (error code=12)\n*** error: can't allocate region\n*** set a breakpoint in malloc_error_break to debug
Sep 27 19:22:52 Macintosh com.apple.launchd[84] ([0x0-0x27027].org.mozilla.thunderbird[315]): Exited abnormally: Bus error


THe problem has gotten worse rather than better. I can be simply composing a new plain text email and while typing I get the spinning color wheel of death. If I just sit and wait, it will either recover (maybe hours later) or crash.

Looking at top, there is no network activity, but a tremendous amount of paging going on.

What can I do to debug this?

Jon K
I do get occasional "hangs" but they don't involve high memory utilization and seem only to occur when switching Wifi connections to another base station.

In most cases these hangs disappear after up to half a minute.

Mac 10.5.5 latest nightly trunk.
Attached file sample of shredder debug in leopard (obsolete) — 
here's a sample of the hang when changing wifi connections. note that gdb doesn't say anything amiss even though |export XPCOM_DEBUG_BREAK=trap|
confirming hang with sample symbols.

also i noted that my real memory usage didn't really keep going up, i only noticed the "virtual memory" figure going up in Activity Monitor.

debug build checkout is Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b1pre) Gecko/20080927233020 Shredder/3.0b1pre but i had also experienced the hang on a latest nightly binary of Shredder optimized.
Status: UNCONFIRMED → NEW
Ever confirmed: true
It crashes, but no crash reports in the Thunderbird Crash Reports directory.

I think what is happening here is that launchd is catching the crash and creating the Apple report. I do not think that Thunderbird is even getting a chance to create a report.

Is there any way to work around this issue and get a Thunderbird generated crash report?

Thanks!

Jon Kibler
Attached file imap protocol log (obsolete) — 
Emre, I managed to get an IMAP protocol log (as I managed to reproduce the hang again).

The locations where the hang occurs (i.e. the spinning Mac bob) are noted in the protocol log.

Is this an issue with Thunderbird or my local servers / settings?
Keywords: qawanted
Hardware: PowerPC → x86_64
Ron, 
Mac crash reporting has improved dramatically, so a nightly should now get you the crash information - http://kb.mozillazine.org/Breakpad#Location_of_crash_reports 

Can you still reproduce?
Version: unspecified → Trunk
I am uncertain if I am having the same problem, but ever since upgrading from thunderbird-2 to thunderbird-3.0 beta2 on Fedora 11, things have been almost unusable.  I have many very large IMAP folders accessed via SSL.  Quite often thunderbird seems to be stuck using 100% CPU without disk activity for 30 seconds or minutes at a time.  Sometimes it recovers on its own and I am able to read or write mail for a short period, but inevitably it begins again within the next few minutes.  Sometimes it seems thunderbird is stuck using 100% CPU and is completely non-responsive for hours.

Multiple other Fedora 11 users are complaining about this issue.
Flags: wanted-thunderbird3?
https://wiki.mozilla.org/MailNews:Logging
export NSPR_LOG_MODULES=imap:5
export NSPR_LOG_FILE=/tmp/imap.log

imap.log grew to 700MB in about 20 minutes.  Curiously, CPU usage of thunderbird remained low and it did not get "stuck" for extended periods of time.

export NSPR_LOG_MODULES=imap:5
export NSPR_LOG_FILE=/dev/null

This workaround seems to be working around most of the problem.  Thunderbird's UI still freezes for a few seconds, but Thunderbird 2 used to do that too.
(In reply to comment #10)
> Multiple other Fedora 11 users are complaining about this issue.

I'm wondering if OpenSuse or Ubuntu folks are getting this, and whether they occur with stock builds.

My comments above turned out to be bug 476960, which has since been fixed.
Different people affected by the bug are you all downloading emails for offline use ?
Warren, I think it would be wise to create your own bug, as the probability is fair that your issue is not the same.  Post your new bug# here.

Gary, why do you think your issue is the same as Jon's?

Jon, 5 months later, I think we need a new read on your problem using the latest nightly. Can you do that with and without lightning?
(In reply to comment #14)
> Gary, why do you think your issue is the same as Jon's?

Which is why I mentioned my issue turned out to be something different altogether. :)
Still have issues.

Large image files (GIF, JPG, PNG) are still problems.

For example, if I want to send a bunch of high resolution images from iPhoto, I generally only send one at a time to prevent the receiving email system from bouncing on too large of an email. However, after about a dozen emails, Thunderbird starts getting slower and the amount of available RAM starts to drop really quickly with each additional message sent (or, sent email viewed). Performance slows to a craw, until finally it simply crashes with a "Bus Error." 

Also, I am still not getting any crash dumps.

Wish I could give you better news.

Jon K
OK, my bug seems similar in that it is IMAP SSL with large folders, but memory usage remains constant and it has never crashed.  I am filing a separate bug.
Jon, if this is related to bug 462013, as you suggest in that bug, then does this also happen with version 2?  with your configuration of
  About 50 imap mail boxes with local replication.
  Folder compression is turned

is that 50 accounts? or 50 folders?
RE: 462013: I still see that behavior, where it sucks up memory and goes into an I/O frenzy, but eventually unwinds itself. I seem to be able to make that problem better by compressing folders every time I receive, delete, or move more than about a dozen emails. I have basically started compressing folders about every hour, when using Thunderbird.

Also, regarding the question of 50 accounts or 50 folders, it is one account with 50 folders, all 50 replicated locally (so I can read email while on long flights).

Finally, regarding the question of crash vs. hang -- for this specific problem (442674), it always ends in a crash. 

I hope this helps.

Jon K
xref bug 480814
Component: Mail Window Front End → Networking: IMAP
Product: Thunderbird → MailNews Core
QA Contact: front-end → networking.imap
Version: Trunk → 1.9.1 Branch
I have seen this issue with an imap folder that is 380,000 messages. TB3 would try to download all the headers, and hangs. Sometimes it will complain about folderpane.js taking too long. If I tried to move a large number of messages out of the huge folder it will use so much memory it will trigger an OOM event on a machine with 4gb of memory.
Would it be worth to get a protocol log with timestamps ?
Keywords: perf
No, this is happening on the client side - the js folder pane issue is because of all the notifications that get sent for new headers, and count changes, etc.

the move issue has been diagnosed elsewhere, especially if you're using drag drop instead of the msg move menu.
(In reply to comment #23)
> No, this is happening on the client side - the js folder pane issue is because
> of all the notifications that get sent for new headers, and count changes, etc.

bienvenu, are you also describing OP's issue?  or is OP's issue more likely folder related?
I was describing Nathan's issue #c21 

Gary's issue seems to be bad/dropped network connections. We've made improvement in that area; I don't know if he's still seeing issues.

I don't know what the current situation is for any of these users...
(In reply to comment #25)
> Gary's issue seems to be bad/dropped network connections. We've made
> improvement in that area; I don't know if he's still seeing issues.

My issue was apparently fixed in bug 476960..
Comment on attachment 341405 [details]
sample of shredder debug in leopard

Possibly-related issue (bug 476960) was fixed.
Attachment #341405 - Attachment is obsolete: true
Comment on attachment 342027 [details]
imap protocol log

Possibly-related issue (bug 476960) was fixed.
Attachment #342027 - Attachment is obsolete: true
I an mo longer having this issue anymore either.

my issues were more profile related I believe as I have since rebuilt my profiles from scratch and have no performance or stability issues reading my IMAP accounts (one machine w/ 1 account the other w/ 4). and some folders have as many as 300K messages in them. (and that's only retained message for the past 90 days.. my server purges them after that).
Jon, how's v3.0.1?

I'm fearful  Jon R  will report he still sees this :)
Flags: wanted-thunderbird3?
Whiteboard: closeme 2010-02-11
RESOLVED INCOMPLETE due to lack of response to last question. If you feel this change was made in error, please respond to this bug with your reasons why.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → INCOMPLETE
Keywords: qawanted
See Also: → 462013
Summary: Unstable reading large imaps mail boxes on server → Unstable reading large imaps mail boxes on server - hang with very high CPU utilization and constantly increasing memory utilization
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: