830482 - Some filters stop working due to the presence of other ones

Status: RESOLVED INCOMPLETE

(Thunderbird :: Filters, defect)

Description

[Martin Allen]

User Agent: Mozilla/5.0 (Windows NT 6.0; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0
Build ID: 20130104151925

Steps to reproduce:

I created the following filters:

1. To Contains "[e-mail]"
Content-Type Contains "text/html"
Move Message to "Spam on ..."

2. To Contains "[e-mail]"
Content-Type Contains "text/plain"
Move Message to "Spam on ..."

3. To Contains "[e-mail]"
Content-Type Contains "multipart/alternative"
Move Message to "Spam on ..."

4. To Contains "[e-mail]"
Body Doesn't contain "452847"
Delete Message

5. To Contains "[e-mail]"
Body Contains "452847"
Move Message to "Deal With on ..."


Actual results:

Problem is when 4 and 5 are enabled, 1, 2 and 3 don't work at all. They start working only when 4 and 5 are disabled.


Expected results:

All should work when all are enabled.

Comment 1

[Thomas D. (:thomas8)]

Martin, try this:

go to Tools > Message Filters and do this with all of your filters:

1 select filter rule, EDIT
2 In the "Perform these actions" section, add the following action:
"Stop filter execution"
3 Click OK in "Filter Rules" dialogue to save the change

After that, all filters should work as expected.

The conditions of each filter rule from 1 to 5, are they linked with AND ("match *all* of the following") or OR ("match *any* of the following")?

Comment 2

[Martin Allen]

Hi Thomas,
Thanks for your help, however adding 'Stop filter execution' did not change the behaviour of the rules.

I have them all on 'Match all of the following' e.g.:

http://www.martin2k.co.uk/miscimages/filter.jpg

Thanks,
Martin

Comment 3

[Martin Allen]

I discovered that Thunderbird doesn't like my custom "Content-Type" header. Perhaps due to the hyphen? I have removed this and the filters work (although now I have three filters that do the same thing).

Oddly, when the three attach filters are on their own, "Content-Type" works.

What I really need is a filter that determines whether the message has any attachments and delete if not (as my attach address ideally will only accept e-mails with attachments).

Comment 4

[Martin Allen]

When I say I removed "this", I mean the rule.

Comment 5

[WADA:World Anti-bad-Duping Agency]

(In reply to Martin Allen from comment #0)
> 4. To Contains "[e-mail]"
> Body Doesn't contain "452847"
> Delete Message

Phenomenon of Bug 695671 ?

Is "change from 'Delete Messsage' to 'Move to Trash folder'" possible in your case?

Comment 6

[Martin Allen]

I tried changing from Delete to Move to Trash but this made no difference.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to Martin Allen from comment #0)
> Actual results:
> Problem is when 4 and 5 are enabled, 1, 2 and 3 don't work at all. They
> start working only when 4 and 5 are disabled.
> Expected results:
> All should work when all are enabled.

What do you call by your "1, 2 and 3 don't work"?
  A "mail which has [e-mail] in To: header" ;
    was moved to different folder from one you expect,
    stayed in Inbox although it should be moved to other folder by a rule, 
    was deleted although it shouldn't, ...
When does your "1, 2 and 3 don't work" occur? (upon or by what event/operation)
  Upon new mail arrival/dowbload, By "Run Filters on folder", ...
What is written in filter log for the mail of "1, 2 and 3 don't work"?

(In reply to Martin Allen from comment #6)
> I tried changing from Delete to Move to Trash but this made no difference.

What is your current/actual filter rules definition? (enabled/disabled, AND conditions or OR conditions, when applied, ...)
Message filter rules is held in file named msgFilterRules.dat under mail directory for each account of Tb's Profile directory.
Can you attach the file to this bug? (never paste to this bug, please)

(In reply to Martin Allen from comment #3)
> What I really need is a filter that determines whether the message has any
> attachments and delete if not (as my attach address ideally will only accept
> e-mails with attachments).

Is your problem following?
  Result of filter execution on a mail by your message filter rule definitions
  is different from your expectation,
  instead of "result is diferent from spec of message filter rule of Tb".

Comment 8

[WADA:World Anti-bad-Duping Agency]

(In reply to Martin Allen from comment #3)
> What I really need is a filter that determines whether the message has any
> attachments and delete if not (as my attach address ideally will only accept
> e-mails with attachments).

When multipart/mixed mail what you concern, "message BODY" for Thunderbird's filtering is currently all *TEXT* lines after "null line which is placed after message headers" to line just before "last close boundary specified by boundary parameter of the Content-Type: multipart/mixed header".
i.e.
(a) If attached part is text/xxx and is base64 encoded, string in data stream corresponds to original "52847" is "base64 encoded data for 52847", so it can't be detected by message filter of Tb.
(b) If attached part is binary such as image/xxx, application/pdf, it's usually base64 encoded. If "string after base64 encode of the binary data" contains 7bits-ascii string of "52847", Tb considers it as a string in "message Body text".

Are you looking this problem?

Comment 9

[Martin Allen]

(In reply to WADA from comment #7)
> When does your "1, 2 and 3 don't work" occur? (upon or by what
> event/operation)

1, 2 and 3 don't work while 4 and 5 are enabled otherwise they do.

>   Upon new mail arrival/dowbload, By "Run Filters on folder", ...
> What is written in filter log for the mail of "1, 2 and 3 don't work"?

Filter log contains nothing relevant to 1, 2 and 3 as they are not being applied.

> (In reply to Martin Allen from comment #6)
> > I tried changing from Delete to Move to Trash but this made no difference.
> 
> What is your current/actual filter rules definition? (enabled/disabled, AND
> conditions or OR conditions, when applied, ...)

When they all enabled, 1, 2 and 3 don't function.

> Message filter rules is held in file named msgFilterRules.dat under mail
> directory for each account of Tb's Profile directory.
> Can you attach the file to this bug? (never paste to this bug, please)

It is now attached (since when I first submitted this bug, I have added a couple of other filters).

(In reply to WADA from comment #8)

> When multipart/mixed mail what you concern, "message BODY" for Thunderbird's
> filtering is currently all *TEXT* lines after "null line which is placed
> after message headers" to line just before "last close boundary specified by
> boundary parameter of the Content-Type: multipart/mixed header".
> i.e.
> (a) If attached part is text/xxx and is base64 encoded, string in data
> stream corresponds to original "52847" is "base64 encoded data for 52847",
> so it can't be detected by message filter of Tb.
> (b) If attached part is binary such as image/xxx, application/pdf, it's
> usually base64 encoded. If "string after base64 encode of the binary data"
> contains 7bits-ascii string of "52847", Tb considers it as a string in
> "message Body text".
> 
> Are you looking this problem?

Oddly, the filters do actually work, but only when the 4 and 5 are disabled.

Comment 10

[Martin Allen]

Attachment: Filter rules 23/02/2013

As requested in reply 7

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to Martin Allen from comment #9)
> Oddly, the filters do actually work, but only when the 4 and 5 are disabled.

I assum; 1/2/3=Move to Spam, 5=Move to Deal With.
When does your "1/2/3 don't work when 4/5 enabled" occur?
  (a) 4=Delete, (b) 4=Move to Deleted Messages

When 1/2/3 worked with 4/5 disabled, what is logged in filer log for mail to which 1 or 2 or 3 is applied(and hit)?

When "1/2/3 don't work when 4/5 enabled", where was a mail, on which you are expecting rule of 1 or 2 or 3 is applied(and hit), placed?
  Inbox?

Problem of when? Upon new mail arrival? Or Manual Filter Run?

Phenomenon like next?
When 4=Move to Deleted Messages, a mail is moved to Deleted Messages or Moved to Deal With, instead of your expectation of "Move to Spam"?

If so, phenomenon is "if 4/5 is disabled, mail is moved to Spam by Junk filter(Tb's, or Server's filter)" and "if 4/5 is enabled, mail is moved by 4/5, then Junk filter((Tb's, or Server's filter) doesn't move to Spam", isn't it?
(i.e. 1/2/3 won't fire for the mail in any case)

Or phenomenon like next?
- With 4/5 was disabled, filter was applied upon new mail check, and 1/2/3 hit.
- With 4/5 was enabled,  filter was applied manually, and 1/2/3 didn't hit.

If you add action of "Add tag-N" to each "rule N", you can easily know "a rule is hit or not" by tag shown at header pane or thread pane if tag column is shown with sufficient column width.

Note:
If "Manual Run" of Tb's filter on custom header, and if mail in an IMAP folder is not held in offline-store file by auto-sync, bug 184490 occurs.
And, because auto-sync is enabled by default, entire mail data is usually saved in offline-store file when IMAP folder of Offline-Use=On, and mail data is not saved in offline-store file when Offline-Use=Off. (Folder Properties/Synchronization)
And, "content-type" is a "custom header" in Tb's Message Filter.
And, because actionValue="imap://[e-mail]/Spam" is seen in your filter definition, I guess your message filter is for an IMAP account.

Comment 12

[WADA:World Anti-bad-Duping Agency]

Because you use both "Body filter" and "filter on custom header", and it looks "filter on IMAP folder", you perhaps can experience any of bugs written in bug 199689 comment #17.
However, if IMAP and Offline-Use=Off, I can't understand why you could define "Body filter rule" via Messae Filter Definition UI of Tb, because "Body" is currently hidden at selection list if Offlne-Use=Off.
And, if Offline-Use=On folder, I can't imagine problem other than Bug 806308 which is upon initial fetch only.
And, if IMAP && Offline-Use=On && Bug 806308, Body filter can do nothing, so "disable/enable of 4/5" can't affect on phenomenon if 4=Move instead of 4=Delete.

If "disable/enable of 4/5" is relevant, I can't imagine other than known problem by "action=Delete", but filter rule is already changed to "Move to Deleted Message".

Bug opener, please clearly decribe about, with what filer rule(phenomenon with 4=Delete or 4=Move to Deleted Messages, 4/5 is enabled/disabled), with what environment(IMAP or not, Offline-Use=On/Off), with what condition/operation(filter upon new mail fetch or Manual Run) etc., and result such as "which filter rules was hit, or no filter rule was hit".

Comment 13

[WADA:World Anti-bad-Duping Agency]

If original message filter definitions, all of followings are used,
  Body filter, filter on custom header, action=Delete
So, if message filter rule for IMAP account, when original message filter is used, any of following bugs can occur,
  Bug 806308, Bug 184490, Bug 695671
and, if you use Received column, you can see Bug 402594 additionally.

Do following, please.
(1) surely rule out Bug 695671 by action=Delete from your problem.
(2) rule out Bug 806308 by Body from your problem.
(3) check with condition of "problem of Bug 184490 doesn't occur"(if manual filter run, set Offline-Use=On, set in CustomDBHeader, and so on).
(4) After it, check about Bug 806308 in your problem.
(5) check about Bug 695671 in your problem.

Comment 14

[Wayne Mery (:wsmwk)]

Martin, can you please test this and post your results.
Thanks

(In reply to WADA from comment #13)
> If original message filter definitions, all of followings are used,
>   Body filter, filter on custom header, action=Delete
> So, if message filter rule for IMAP account, when original message filter is
> used, any of following bugs can occur,
>   Bug 806308, Bug 184490, Bug 695671
> and, if you use Received column, you can see Bug 402594 additionally.
> 
> Do following, please.
> (1) surely rule out Bug 695671 by action=Delete from your problem.
> (2) rule out Bug 806308 by Body from your problem.
> (3) check with condition of "problem of Bug 184490 doesn't occur"(if manual
> filter run, set Offline-Use=On, set in CustomDBHeader, and so on).
> (4) After it, check about Bug 806308 in your problem.
> (5) check about Bug 695671 in your problem.

Comment 15

[Phoenix]

Resolved per whiteboard