230672 - IMAP: compacting folders (IMAP purge) takes long time

Status: RESOLVED INVALID

(MailNews Core :: Networking: IMAP, defect)

Description

[Calum Mackay]

Starting with my nightly build of 2004011105 (checkout time 0500 GMT) I am
seeing considerable delays in purging IMAP folders.

If I look at a folder with a dozen or so messages in it, mark 6 for deletion,
and then compact the folder (i.e. imap purge), it takes about 45 seconds, along
with much continuous imapd cpu activity on the server (which is the same system,
in my case), for both my nightly CVS build yesterday of 2004011105 and today's
build 2004011205. However, using my previous build 2004011005, the compaction is
almost instantaneous.

As far as I can see, the folder is compacted correctly, so this is just a [big]
performance problem, as opposed to a functional problem.

My imap server is UoW imap-2002e (running over SSL):

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4REV1 LOGIN-REFERRALS STARTTLS LOGINDISABLED] localhost
IMAP4rev1 2003.339 at Mon, 12 Jan 2004 09:03:26 +0000 (GMT)

Both client and server (same system) are Debian Linux/x86. I'm seeing this in
both my mozilla and Thunderbird builds (unsurprisingly)

Comment 1

[Hermann Schwab]

from your comment #0 it seems to be regressed between 2004011005 and 2004011105

maybe from this checkin?

01/10/2004 15:48 fix imap compact all to compact all imap folders, and imap
compact to compact offline stores, Bug 222938

Comment 2

[Calum Mackay]

Interestingly, I don't see the problem being as bad when I point the same builds
at my work imap server. The differences are that the work server (which doesn't
seem to see the problem as much) is:

o A much faster multi-cpu Solaris system; there is about a 5 second delay on
this system, which is still [possibly] longer than normal.

o Running an older uw-imapd:

* OK [CAPABILITY IMAP4REV1 LOGIN-REFERRALS STARTTLS AUTH=LOGIN] clem IMAP4rev1
2003.337 at Mon, 12 Jan 2004 10:58:35 +0000 (GMT)

o Running plain imap, not imap/ssl.

o Running via a compressed ssh tunnel

Comment 3

[Calum Mackay]

Aha! I've just seen comment 1; very interesting. I have lots of folders, so this
might well be relevant...

Comment 4

[Calum Mackay]

Yes, that's it, exactly. compact folders is indeed now compacting *all* folders,
and so it takes a while. I could not reproduce against my work account, owing to
it having less folders.

the folder context compact just compacts the current folder, and that is as
quick as it ought to be.

marvelous, thanks for the pointer; this might be a candidate for release notes
though, perhaps?

Comment 5

[David :Bienvenu]

No good deed goes unpunished :-)

Release note: File | Compact Folders for imap accounts now compacts all imap
folders in the account, and their offline stores, if any folders are configured
for offline use. Previously, this command just compacted the selected folder. To
compact just the selected folder, you should now use the folder context menu
command Compact this Folder.

Comment 6

[Calum Mackay]

thanks much David :)

Comment 7

[Calum Mackay]

I note something odd: if I do File | Compact Folders, it only compacts all
folders if the current folder is something other than my inbox. If the current
folder is my inbox, then only the inbox is compacted.

Is this the intended behaviour?

Comment 8

[Calum Mackay]

Ok, scratch that. The async nature of it makes it look different, but it's not.

If my inbox is the current folder, I see it compacted *immediately*, and other
folders are compacts subsequently, in the background.

If another folder is the current folder, I do not see it compacted immediately.

I imagine the algorithm is to start with inbox, and then iterate over all other
folders.

As an ease of use issue, would it be better (or possible) to start with the
*current* folder, rather than inbox [unless current *is* inbox], and then
iterate over the others?

from the view of user feedback, this might be preferable.