584288 - Filters don't work all the time - some messages stay in Inbox

Status: RESOLVED INCOMPLETE

(MailNews Core :: Filters, defect)

Description

[jdl]

User-Agent:       Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; InfoPath.2; .NET CLR 3.5.21022; .NET CLR 3.5.30729; .NET CLR 3.0.30618; OfficeLiveConnector.1.5; OfficeLivePatch.1.3)
Build Identifier: 3.1.1 and on....

I have filters set up in my mail account and since 3.1.1 this functionality is half broken. A couple of messages are moved to the folder designated by the filter but most stay in the inbox. When I click Apply filters to folder then the messages are moved after that but I do have to do that manually each time I receive mail now

Reproducible: Always

Steps to Reproduce:
1. Set up filters on multiple email accounts to one main account's inbox
2. Set up filters from the main account inbox to subfolders you want them to be
3. Get email and notice that not all filters are applied automatically
Actual Results:  
Get email and notice that not all filters are applied automatically, than apply filters to folder manually

Expected Results:  
At initial mail receice not all messages are moved to the correct folder. After manually pressing apply filers to folder they will.

Apply the filters to all the messages when the email is received from the server. In previous versions this was always the case.

Comment 1

[Kent James (:rkent)]

This needs to be narrowed down from "not all filters" work to specific examples of filters that work and don't work. Also, is that behavior reproducible or intermittent?

Comment 3

[Wayne Mery (:wsmwk)]

are any filters on message body?
any recent changes to any filters?

Comment 4

[jdl]

No only From is and From contains and a couple of subject is and subject contains.

To my knowledge I changed nothing to the filters. The only thing I remember was changed was upgrading to the latest version of Thunderbird which was 3.1.1 and is now 3.1.2. I did notice however that newly added filters are now shown as From is: emailadress instead of Name (which I always named the filterpart of the filter). Also I noticed a new filter 'All messages' which I apply now from one of my accounts (info@...) to my main account (fastucom). I deleted all other filters in that account. That's about all latest changes I can think of now.

Comment 5

[Wayne Mery (:wsmwk)]

jdl, do you still see this problem when using current version?

Comment 6

[jdl]

Well, I've removed all my filters in the mean time and created all new filters on the inbox so it would work again.

Comment 7

[jdl]

Actually, I noticed that the problem stil exists. Using TB 6.0 now.

Comment 8

[:aceman]

Could you attach the file msgFilterRules.dat from you mail folder?

Comment 12

[:aceman]

Do you have the filters in the correct order (as they should apply)? In the "msgFilterRules.dat 2 and 3" files you have "ALL move to Inbox" as the last rule. Is this also the order in which they appear in the UI (tools-> message filters?). Can you try moving it to the top?

Comment 13

[jdl]

No that is correct. I want all messages that not already have filters to be rerouted to my main Inbox instead of the Inboxes of those email addresses itself. So that is only applied for the messages that do not have a filter yet.

Comment 14

[:aceman]

So which file contains the rules for the main Inbox/account?

Comment 17

[jdl]

The filters are already on "When Checking Mail (after classification) or Run manually" so this doesn't help.

Comment 18

[WADA:World Anti-bad-Duping Agency]

(In reply to jdl from comment #0)
> Steps to Reproduce:
> 1. Set up filters on multiple email accounts to one main account's inbox

What is server type of the main account?
POP3 account? IMAP account? Or pseudo account of Local Folders"?

There is known problem like next;
  - If filter action such as "move to folder" somehow fails, it silently fails.
  - If contention on move target folder happens, move silently fails.
"Silently" in this context means:
  Filter log says "filter was applied", but "move to folder" was not executed.

If move target folder of filter action is Inbox of an active account, "download of mails to Inbox of the account by periodical new mail check" and "filter move by filter of other accounts to Inbox of the account" occurs at same time. It produces contention of the Inbox folder of the account. This may cause silent filter move failure.
Because manual "Run Filters on Folder" works well, and because your filter rule/action is simple(simple if condition with simple move to folder action only), your message filter looks written correctly and problem like "false positive or false negative of filter condition" doesn't look to happen in your case.
So, "silent failure due to Inbox busy" is suspected in your case. 

What is reason to move new mails of other accounts by filter to Inbox to which mails are frequently downloaded at same time?
What is you setting of "check new messages every NN minutes" of all accounts?
Do you see your problem with filter action of "move to non-busy folder of main account"? (e.g. move to Inbox2 of the main account)

FYI.
Contention of Inbox folder due to "intentional use of such busy Inbox as folder to save sent mail copy by user" is known. In this case, save to sent folder(==save to Inbox) frequently fails due to contention, then Retry/Cancel dialog for sent mail copy is frequently shown when mail is sent.

Comment 19

[WADA:World Anti-bad-Duping Agency]

FYI.
Other example of problem due to contention of folder.
Junk move upon mail download sometimes fails if auto-compact is invoked on Junk folder at same time. Because "deletion of mails" usually happens on Inbox by filter move and junk move upon mail download, auto-compact is frequently invoked while downloading of mails than we think if small auto-compact threshold is specified by user and if "dialog before start of auto-compact" is killed by user.
If big Inbox, compact of Inbox takes long. So, this kind of auto-compact may interfere "move mails to the Inbox by message filter of other account" too.

Comment 20

[jdl]

The account is POP3.
I've adjusted the threshold for compacting folders from 1MB to 11MB now, perhaps that will make a difference.
In what log file can I verify that the filters where applied succesfully or not?

Comment 21

[:aceman]

Do you get the error "aFolder is undefined" in Tools->Error console after the filters were run?

Comment 22

[Wayne Mery (:wsmwk)]

jdl ?

Comment 23

[Wayne Mery (:wsmwk)]

marked comments/attachments with email addresses as private for confidentiality

Comment 24

[Wayne Mery (:wsmwk)]

(i missed a couple of the comments)

Comment 25

[jdl]

The problem is still there in version 11.
I'm now moving the filters to the mail addresses the mail gets first into instead of first forwarding it to the main inbox and then filtering it again to my subfolders. That works, it is some more work though.

Comment 26

[Wayne Mery (:wsmwk)]

jdl?

(In reply to :aceman from comment #21)
> Do you get the error "aFolder is undefined" in Tools->Error console after
> the filters were run?

Comment 27

[Wayne Mery (:wsmwk)]

other "filtefails" bugs https://bugzilla.mozilla.org/buglist.cgi?type1-0-0=substring;list_id=2774716;field0-0-0=short_desc;type0-0-1=substring;field0-0-1=keywords;type1-0-1=allwordssubstr;resolution=---;classification=Client%20Software;classification=Components;status_whiteboard_type=allwordssubstr;query_format=advanced;status_whiteboard=fail;type0-0-0=anywordssubstr;component=Filters;field1-0-0=short_desc;product=MailNews%20Core;product=Thunderbird;field1-0-1=short_desc

Comment 28

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 29

[Wayne Mery (:wsmwk)]

we still need this piece of info

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #26)
> jdl?
> 
> (In reply to :aceman from comment #21)
> > Do you get the error "aFolder is undefined" in Tools->Error console after
> > the filters were run?