388347 - Problems expunging ~100 messages in a folder - compact this folder hangs high cpu

Status: RESOLVED DUPLICATE of bug 218075

(MailNews Core :: Networking: IMAP, defect)

Description

[Scott MacGregor]

Matthew was telling me about a problem he is having with Thunderbird 2.0.0.4 using the mozilla.com IMAP server. He's on a Mac.

Delete model is: mark as deleted.

Over the course of the day he marks a hundred messages or so as deleted. He'll then manually compact folders and Thunderbird appears to hang. But if you force quit tbird and restart, the messages have been deleted from the server.

Even on windows he notices the CPU spiking but the operation does complete.

Comment 1

[Scott MacGregor]

Matthew, when you compact folders are you doing File / Compact Folders or are you using the folder specific compact folder operation available via the context menu in the folder pane?

Comment 2

[matthew zeier [:mrz]]

I use File | Compact Folders.

Is there anything I can turn on to catch it failing?  

Comment 3

[David :Bienvenu]

that compacts *all* your folders. If you just want to compact a single folder, use the context menu on the folder to issue the expunge...

Comment 4

[David :Bienvenu]

it might be a particular folder that's slow to compact - you could try doing them one at a time and see if one of them causes the lockup...

Comment 5

[Scott MacGregor]

Matthew, any luck using the compact folder context menu option as a way to figure out which specific folder is causing the problem?

Comment 6

[matthew zeier [:mrz]]

I've been trying but haven't been able to find a problem folder.  Only happens with I compact all folders.

Comment 7

[matthew zeier [:mrz]]

Found the folder.

I have system generated emails (from root, cronjobs and other system noise) filtered into INBOX/root.  This folder is -not- set to check for new messages.

It's not uncommon to have more than 500 messages at a time when I do look into that folder.  Some are marked as deleted from a session on another computer.  If I compact this folder, tb hangs.  This used to happen on Windows but I'm on OSX and it happens here too.  

Comment 8

[matthew zeier [:mrz]]

Failed to mention that after I forcibly kill Thunderbird and restart, those messages that were marked for delete are actually expunged.

Comment 9

[David :Bienvenu]

so, if you do a context menu | compact this folder, on this particular folder, it hangs every time? 

Comment 10

[matthew zeier [:mrz]]

Nearly every time, yes.

Comment 11

[Wayne Mery (:wsmwk)]

mrz, can you reproduce with latest beta?
http://www.mozillamessaging.com/en-US/thunderbird/early_releases/ ... backup
your profile before using

Comment 12

[matthew zeier [:mrz]]

Unfortunately I stopped using Thunderbird and am no longer a good test case.

Comment 13

[Wayne Mery (:wsmwk)]

needs test with mark as delete?  (i don't use mark as delete)

Comment 14

[David :Bienvenu]

I think Neil might use the mark as deleted mode.

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #7)
> Found the folder.
> I have system generated emails (from root, cronjobs and other system noise)
> filtered into INBOX/root.  This folder is -not- set to check for new messages.
> It's not uncommon to have more than 500 messages at a time when I do look into
> that folder.  Some are marked as deleted from a session on another computer. 
> If I compact this folder, tb hangs.  This used to happen on Windows but I'm on
> OSX and it happens here too.

It sounds Bug 452221.
(a) cronjobs, so many mails has same subject, then very long thread happens. 
(b) "deleted by other client", so similar .msf update to "Shift+Delete of mails"
    can occur upon folder open or EXPUNGE by Compact or Compact Folders.
(c) If "delete by other client" is executed multiple times,
    number of EXPUNGED mails can become large.
(d) If "Mark as deleted" model, "remove of many mails from long thread" can
    usually occur only after EXPUNGE by Compact. (Bug 452221)

Problem only when "Mark as deleted"? 
I guess similar phenomenon can occur upon folder open when "Move to trash" or "Remove immediately" model is used. It can occur upon folder open even when "Mark as deleted", if many mails are deleted and expunged by other client before Tb opens the mail folder.
Is it wrong guess?

Comment 16

[Wayne Mery (:wsmwk)]

wada's comment 15 sounds logical. but bug 218075 sounds like a closer dup (which notes that performance might be even worse in the gloda indexing environment)