193325 - multiple mail check/filtering instances cause mail multiplication

Status: RESOLVED DUPLICATE of bug 381588

(MailNews Core :: Filters, defect)

Description

[ast]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021203
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2) Gecko/20021203

When moving mails from a remote IMAP Inbox to a local IMAP folder via
filter(s) and "check for new mail every 'n' minutes" over a slow remote
connection messages are duplicated (triplicated, ...) if the processing
time (filtering/moving) for the contents of the remote Inbox is longer
than the time set in "check for new mail every 'n' minutes".


Reproducible: Always

Steps to Reproduce:
1. Create two IMAP accounts, one local (LAN) and one remote (WAN or any other
   slow network connection)
2. Assert that Mozilla stores the account passwords to prevent possible
   timing issues by improper password input for the following sequence
   (a master password for all account passwords is fine)
3. Setup Mozilla to check for new Mail on the remote IMAP account every
   minute and to check for new Mail on the remote IMAP account on startup
4. Create filter(s) that will move the mails from the remote IMAP Inbox to
   some folder of the local IMAP account.
5. Shutdown Mozilla
6. Fill the remote IMAP Inbox with sufficient mails so that the processing
   by Mozilla takes definitely longer than one minute, e.g. two minutes
   should be fine.
7. Start Mozilla, enter the master password if required and watch

Actual Results:  
Mails show up in the destination folder(s) multiple times, this looks like
the timed 'look for new mail and filter' job is started regardless if the
previous job hasn't yet completed.


Expected Results:  
Only one instance of every filtered mail in the destination folder, i.e.
serialization the IMAP mail folder processing.


This also happens when a manual folder open of the remote Inbox collides
with the automatic check for new mail, though this constellation is
quite difficult to achive.

Comment 1

[ast]

The junk filter in Mozilla 1.3 now helps to trigger this problem reliably.
This means: instead of manually filtering about 25 to 50 SPAMs per day
I do now need to manually delete about 100 to 200 duplicate mails per day.
This is now more than a nuisance and  causes me to change the severity of
this bug to major.

Comment 2

[ast]

With actived junk mail filtering of Mozilla 1.3 (training.dat size is 16MB)
I do get even triplicates over a 100MBit LAN connection.
This problem is really severe.

Comment 3

[Mike Cowperthwaite]

*** Bug 193021 has been marked as a duplicate of this bug. ***

Comment 4

[Mike Cowperthwaite]

Likely a duplicate of bug 167173.

Comment 5

[ast]

Unlikely a duplikate of 167173 as the duplicates don't appear in the Inbox.
They do appear in the target folder. And they are not marked as deleted.

Comment 6

[Christian Hamacher]

Probably a duplicate of Bug 176691 - while there, the problem appeared when
using the 'Move To' option from the context menu, this probably triggers the
same code to move the message - and the symptoms are exactly the same.

Comment 7

[Asa Dotzler [:asa]]

*** Bug 214297 has been marked as a duplicate of this bug. ***

Comment 8

[Mike Cowperthwaite]

*** Bug 304735 has been marked as a duplicate of this bug. ***

Comment 9

[Mike Cowperthwaite]

*** Bug 327268 has been marked as a duplicate of this bug. ***

Comment 10

[ast]

I don't currently konow about trunk, currently I'm using Thunderbird 2.0.0.9. I just left offline mode and selected INBOX on a DSL 16000 (16000 kBit downstream, 1000 kBit upstream) line. Thunderbird decided to use 100% CPU for more than one full minute to process 87 mails and the result is again duplicate eMails all over the place. Let's see if Bugzilla allows me to attach a screenshot :)

Comment 11

[ast]

Attachment: Duplicates served by thunderbird 2.0.0.9

Duplicates served by thunderbird

Comment 12

[spamtrap]

This bug is still present in 2.0.0.14 and can get quite ridiculous with upto hundreds of duplicates for those of us who try and use thunderbird for corporate (i.e. heavy) use.  The lack of a target milestone is disconcerting for a major bug, either this is a major bug and should be addressed on the roadmap somewhere or or it isn't considered a major bug and should be downgraded.  

If it is the latter, then it should be made clear, so that those of us trying to use TB in such environments can swap to another client.  I don't mean this as a criticism of TB, it's simply allowing users to understand the Projects priorities.

Comment 13

[David :Bienvenu]

I have a vague recollection of fixing something like this on the trunk (e.g., 3.0 a1), but I could be wrong. I'll try to go back and refresh my memory as to what it was, but if you're feeling adventurous, spamtrap, you might try 3.0 a1.

Comment 14

[David :Bienvenu]

see bug 381588 - I believe the fix there would help with this issue. I never landed it on the 2.0 branch; I can try to get approval for that.

Comment 15

[Wayne Mery (:wsmwk)]

spamtrap, ast, can you try trunk build ftp://ftp.mozilla.org/pub/thunderbird/releases/3.0a1/ to determine if problem is gone or reduced?  You'll want to install in it's own directory.

Comment 16

[spamtrap]

I know this sounds like a ridiculous question but how alpha is the alpha?  I'm normally more than happy to help out but I confess I'm reticent to run alpha software on something as important as corporate mail.

Comment 17

[Wayne Mery (:wsmwk)]

with few exceptions, mostly voluntary, I have run trunk for the last two years.

Comment 18

[David :Bienvenu]

the next update to 2.0, 2.0.0.15, will contain this fix as well. It should be coming out pretty soon (a matter of weeks, I believe)