Compact of IMAP folder with many deleted message slow with IMAP delete model (Mark as Deleted) because of expunge notification from the server for every deletion

NEW
Unassigned

Status

MailNews Core
Networking: IMAP
--
major
15 years ago
3 years ago

People

(Reporter: Bienvenu, Unassigned)

Tracking

(Blocks: 2 bugs, {perf})

Dependency tree / graph
Bug Flags:
blocking-thunderbird3 -
wanted-thunderbird3 +

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [bulkoperations])

(Reporter)

Description

15 years ago
As reported by Neil, if you're using the imap delete model, and have an imap
folder with lots of deleted messages, when you compact the folder, the command
takes a long time to finish (in the reported case, for 4000 messages, it took 10
minutes).  This is always going to be somewhat slow because we're going to get
4000 expunge notifications from the server, but we can speed up the UI part of
this by batching updates to the outliner.

The code in question is in nsImapMailFolder::UpdateImapMailboxInfo:
    if (keysToDelete.GetSize())
    {
      PRUint32 total;
      
      // It would be nice to notify RDF or whoever of a mass delete here.
      if (mDatabase) 
      {
        mDatabase->DeleteMessages(&keysToDelete, nsnull);
        total = keysToDelete.GetSize();
      }
    }

Unfortunately, this code knows nothing about the view - we would either need to
write some hacky code to get the view and do batching, or add some sort of batch
notification between the db and the view (or whatever listeners were open on the
db). The latter is much more general.
Product: MailNews → Core

Comment 1

13 years ago
This is my first time submitting a bug... not sure if this is the right place. 
I've noticed "Compact Folders" is very slow on win2k.  My INBOX is about 8 Mb. 
I just deleted 1 message and compacted the folder.  It took about 3 minutes.  My
incoming server is imap and I have "When I delete a message" set to "Mark it as
deleted".  I've noticed this on multiple machines with different configurations.
 Also, I've talked to other people with the same issue.  I've noticed this since
Thunderbird ~0.9.  I am now using 1.0.  I do not get this problem with Linux.

Comment 2

12 years ago
does the general theory also apply to Bug 296453?
(Reporter)

Comment 3

12 years ago
maybe - I think for most operations we do batch the updates - but maybe not for the imap delete model.

Updated

10 years ago
Severity: normal → major
Keywords: helpwanted
QA Contact: grylchan → networking.imap
(Assignee)

Updated

9 years ago
Product: Core → MailNews Core

Comment 4

9 years ago
comment #2
> does the general theory also apply to Bug 296453?
(In reply to comment #3)
> maybe - I think for most operations we do batch the updates - but maybe not for
> the imap delete model.

bienvenu, do we still need someone to investigate this? 

(bug 296453 is dealing with front end)
Keywords: perf
Whiteboard: [batch operation]

Comment 5

9 years ago
Thunderbird[1] "freezes" for (seemingly endless) seconds when I manually compact my primary account's IMAP inbox - even if there is only one marked-as-deleted message there. The whole UI is frozen until the compact is done. Oddly, this doesn't occur on the other IMAP sub-folders nor (IIRC) on my other IMAP inboxes.

BTW: Ever since Tb went to a different storage format, there are all kinds of circumstances (seemingly all message interactions) where Tb becomes unresponsive to to messages indexing/hard-drive-thrashing/etc.

[1] Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4pre) Gecko/20090829 Lightning/1.0pre Shredder/3.0b4pre

Updated

9 years ago
Flags: blocking-thunderbird3?
(Reporter)

Comment 6

9 years ago
Tb hasn't gone to a different storage format.

But Gloda might be listening for imap compaction events...
(Reporter)

Updated

9 years ago
Flags: wanted-thunderbird3+
Flags: blocking-thunderbird3?
Flags: blocking-thunderbird3-

Updated

8 years ago
Duplicate of this bug: 388347

Comment 8

7 years ago
bienvenu, given other recent batching successes, is there something we can easily do here?
(Reporter)

Updated

6 years ago
Assignee: dbienvenu → nobody

Updated

5 years ago
Blocks: 498274
Keywords: helpwanted
OS: Windows 2000 → All
Summary: Compact of IMAP folder with lots of deleted message slow with IMAP delete model → Compact of IMAP folder with many deleted message slow with IMAP delete model because of expunge notification from the server for every deletion
Whiteboard: [batch operation] → [bulkoperations]

Updated

3 years ago
Summary: Compact of IMAP folder with many deleted message slow with IMAP delete model because of expunge notification from the server for every deletion → Compact of IMAP folder with many deleted message slow with IMAP delete model (Mark as Deleted) because of expunge notification from the server for every deletion

Updated

3 years ago
Duplicate of this bug: 555171
You need to log in before you can comment on or make changes to this bug.