277042 - Message filter processing of IMAP inbox very very slow

Status: RESOLVED EXPIRED

(Thunderbird :: Untriaged, defect)

Description

[Andy Romero]

User-Agent:       Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
Build Identifier: version 1.0 (20041206)

Background:
In the morning when I login to my IMAP server,
there will be about 1000 new messages waiting for
me. Approx 980 of these messages are to be
processed by message filters which move
the messages from the IMAP server to local mail folders.

PROBLEM
The filters never (I have waited 20min)
finish processing and I can not access messages
in my IMAP inbox.

If I un-check all the message filters 
(so they don't auto execute)
and I run them one-by-one using the RUN-NOW button,
the filters process the IMAP inbox quite quickly.

When the filters are checked for Auto execution,
is each message filtered as it's cached or
are all messages cached first ?

Note: My IMAP server is a Solaris box
      running Sun Java System Messaging Server 6.1 HotFix 0.02
     (built Aug 25 2004)






Reproducible: Always

Steps to Reproduce:
see details
Actual Results:  
see details

Expected Results:  
see details

Comment 1

[Ansgar Schrage]

I have the same effect when getting mail from a POP3 account. I have about 200
emails waiting to be downloaded that have to be filtered also. It takes almost a
second for each mail to get into my inbox. I guess that the filter component has
some decent performance problems.
The filter that is the slowest moves the incoming message to a certain folder.

Comment 2

[Gervase Markham [:gerv]]

This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/

Comment 3

[Gervase Markham [:gerv]]

This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.

Comment 4

[József Kószó]

This filter problem is still exists in recent Thunderbird version 24.0 on Ubuntu Linux (probably OS is irrelevant).

I've daily hundreds of emails and ten automatic filters (15-20 rules), which moves mails to several subfolders. Automatic filtering of incoming messages is extremely slow, but if I close Thunderbird (while filtering) and start it again (the result is many unreaded/unfiltered mail in incoming folder), and manually select run filters on folder from the menu, then all messages are filtered in seconds magically, and moved to corresponding folders, while the activity viewer show the filtered result:

  45 msg to admin
  37 msg to ntpdate
  852 msg to zabbix
  121 msg to snmp
  27 msq to qa

then Thunderbird indexing the changed folders. The root of this bugreport is the automatic filter, which filtering incoming messages one by one (see activity viewer), applying all rules, then actualizing and indexing the target folder, and starts next round. The manual folder filter takes all messages, and filtering in one round, thus mass move all messages to its subfolders.

Suggestion: automatic incoming message filtering should not start while messages/headers downloading is in progress, or must filter all downloaded new messages, not just one by one.

Comment 5

[Wayne Mery (:wsmwk)]

(In reply to k.joe from comment #4)
> This filter problem is still exists in recent Thunderbird version 24.0 on
> Ubuntu Linux (probably OS is irrelevant).
> 
> I've daily hundreds of emails and ten automatic filters (15-20 rules), which
> moves mails to several subfolders. Automatic filtering of incoming messages
> is extremely slow, but if I close Thunderbird (while filtering) and start it
> again (the result is many unreaded/unfiltered mail in incoming folder), and
> manually select run filters on folder from the menu, then all messages are
> filtered in seconds magically, and moved to corresponding folders, while the
> activity viewer show the filtered result:

I'm not sure it's possible to have messages in your inbox that haven't been filtered.
But, that aside...

If you can reproduce the problem **in safe mode** (see steps below), please file a new bug report with https://bugzilla.mozilla.org/enter_bug.cgi?product=Thunderbird&component=Filters


Start thunderbird in safe mode per http://support.mozillamessaging.com/en-US/kb/safe-mode

Does problem go away?

- If no, then cause is either: bug in Thunderbird, in your Thunderbird profile/folders, or your mail provider. Please provide info from Help | Troubleshooting | copy text to clipboard

- If yes that problem is gone, then cause is likely a Thunderbird addon - eliminate them by disabling each one at a time in Tools | addons | extensions and restarting

Comment 6

[József Kószó]

The problem is not the unfiltered messages, but the slow (one by one) filtering while downloading new messages (see the first two comments above). Messages just remains in my inbox, because the downloading/filtering intentionally interrupted by closing Thunderbird, then I can run filters manually, which is significantly faster. I tried safe mode and/or decrease parallel IMAP connections with previous versions, but I'll check it again, and file a new report.

Comment 7

[József Kószó]

See also: https://bugzilla.mozilla.org/show_bug.cgi?id=971401