Shredder starts downloading All Mail, periodically hangs eating CPU

RESOLVED DUPLICATE of bug 519543

Status

Thunderbird
General
--
major
RESOLVED DUPLICATE of bug 519543
9 years ago
9 years ago

People

(Reporter: Simon Marlow, Unassigned)

Tracking

({perf})

x86
Windows XP

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [fixed by bug 519543])

(Reporter)

Description

9 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)
Build Identifier: 3.0pre 20091018

I downloaded a nightly to get the fix for #517256.

Shredder apparently thinks my GMail "All Mail" folder needs to be downloaded and indexed.  That wouldn't be so bad, except that while doing so it periodically becomes unresponsive for 30 seconds or so, apparently while indexing messages (according to the activity monitor).  When it wakes up it will be responsive for a short while before locking up again.  After several hours it had only managed to download and index around 10% of the 140k emails in All Mail.

Going back to 3.0b4, and the All Mail folder does not need to be re-downloaded.  "check for new messages" is not checked, but the folder is marked for offline use.


Reproducible: Always
Simon you are using imap right ?

can you produce an imap log when the issue arise ?
(Reporter)

Comment 2

9 years ago
I started up Shredder again with IMAP logging enabled.  It continued to download by All Mail folder, and gradually got less and less responsive.  While it wasn't hanging for 30+ seconds as in my previous report, by the time I killed it it was becoming unresponsive for several seconds at a time, which made it essentially unusable.

The log contains various personal info, so I don't want to attach it here.  What do you suggest?
(In reply to comment #2)

> The log contains various personal info, so I don't want to attach it here. 
> What do you suggest?

Anonymize it, remove the names and addresses ...
(Reporter)

Comment 4

9 years ago
(In reply to comment #3)
> (In reply to comment #2)
> 
> > The log contains various personal info, so I don't want to attach it here. 
> > What do you suggest?
> 
> Anonymize it, remove the names and addresses ...

It is 190MB.  Do you have a script for anonymizing these logs?
NO :-( but some simple search replace should do most of it.

Comment 6

9 years ago
you should update again to latest nightly, and turn off indexing until more gloda patches land.

Updated

9 years ago
Keywords: perf
Version: unspecified → 3.0

Comment 7

9 years ago
I'm also seeing a very similar issue in: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a1pre) Gecko/20091116 Shredder/3.1a1pre

Connect to a large IMAP mailbox (more then a few thousand messages in each folder), tell TB to download message in this folder for offline use.  While TB is downloading, the global indexer will be attempting to index.  CPU usage spikes to 100% of one of the system's CPUs, UI responsiveness goes to hell and hangs every few seconds for a few seconds at a time.

Turning off global indexing (gloda) does seem to help.

Comment 8

9 years ago
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091117 Thunderbird/3.0

Seems to be mostly fixed for me in TB 3.0 RC 1 build 2.  I've been told that 1.9.3 won't pickup the fix until Bug 519543 lands (and possibly Bug 528802).

I'm still seeing 1/10th second pauses now and then if gloda is working on indexing, so it's not perfect.
(Reporter)

Comment 9

9 years ago
I've installed 3.0RC1 build 2, and indeed it is much better.  I'm still seeing multi-second hangs occasionally, but much less frequently than before.

Comment 10

9 years ago
awesomeness. So I think we can say this is fixed by bug 519543. please correct the bug# if i've got it wrong.

the shorter freezes I see also, to be covered in other bugs
Status: UNCONFIRMED → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
Whiteboard: [fixed by bug 519543]

Comment 11

9 years ago
The freezes that I see in 3.0RC1 build 2 are (I think) due to TB downloading messages, or at least I'm seeing more frequent freezes of a few seconds at a time whenever TB is pulling messages down.

Seems to be freezing at the end of keeping an IMAP folder up to date.  So maybe whenever it finishes a folder, it locks up for 10-15 seconds at a time.
The most likely explanation for a lock-up when running 3.0rc1 build 2 is that the synchronous .msf opens are expensive.  So rather than the freeze happening at the end of folder "A", it would happen at opening folder "B" which would happen immediately after folder A and so look similar to what you describe.  This would happen in a folder that has *lots* of messages.  For example, gmail folders can end up with a _lot_ of messages in them.

If you could open up the activity manager (from the tools menu) and see what the active/recently active activities were and whether they involve folders with lots of messages, that would be very helpful.
Status: RESOLVED → UNCONFIRMED
Resolution: FIXED → ---

Comment 13

9 years ago
duping to bug 519543.
for other issues (eg comment 12 et al), please open a new bug, or comment in an existing one
Status: UNCONFIRMED → RESOLVED
Last Resolved: 9 years ago9 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 519543
You need to log in before you can comment on or make changes to this bug.