Open Bug 435286 Opened 16 years ago Updated 2 years ago

Message filter(to or cc,contains) does not work properly, if no parameter data in To: header and multiple CC: headers

Categories

(MailNews Core :: Filters, defect)

x86
Windows XP
defect

Tracking

(Not tracked)

People

(Reporter: herter, Unassigned)

References

Details

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14
Build Identifier: Thunderbird/2.0.0.14

Since of the recent Thunderbird versions numerous messages appear in my inbox, which were previously for years reliably filtered out into a dedicated mailbox. Even when I create a new message filter based on one of such messages, the new seemingly redundant filter will be ignored and some more messages appear in my inbox. 

The filter in question is "if to or cc contains 'clusterwr', move message to LocalFolders/Clusterware." And yet numerous messages will not be moved over, but some are. SO far I found no obvious differentiator. This is an example header of a message which was not moved into Clusterware: 

From: JOHN.LEYS@ORACLE.COM
To: 
CC: CLUSTERWR_US@ORACLE.COM, DENISE.IGUMA@ORACLE.COM, etc. etc..
CC: TAK.WANG@ORACLE.COM

And this is one which will be filtered properly:

From: HARSHA.KANCHARATHI@ORACLE.COM
To: 
CC: CLUSTERWR_US@ORACLE.COM, HARSHA.KANCHARATHI@ORACLE.COM, etc. etc..
Date: Wed, 14 May 2008 12:14:10 -0700

(its very difficult to get the header via cut/paste with Thunderbird, its a real step backward compared to Netscape Messenger... Maybe a screen dump would help, or maybe I could open up the text file using editor and get the header??
I used 'Forward as /Inline' to get a selectable text of the header)

Maybe the second CC: is the culprit, maybe it overwrites the check of the first one?




Reproducible: Always

Steps to Reproduce:
1.Send a message with a to or cc containing filter str, like clusterw in my case.
2.Have a filter with 'to or cc contains that string' and action 'move to a folder'
3.
Actual Results:  
Stays in inbox

Expected Results:  
Expected to be in local folders/Clusterware
> Maybe the second CC: is the culprit, maybe it overwrites the check of the first
one?

Following is my test result of duplicated headers case, with local mail folder(mail folder for dummy POP3 account).
Problem on "to or cc,contains" when multiple CC: headers couldn't be re-produced.

>(Mail source. Triple headers of Date:,From:,To:,CC:,Subject:)
> Date: Sat, 01 Jan 2001 01:01:00, & Date: Sat, 02 Jan 2002 02:02:00 & Date: Sat, 03 Jan 2003 03:03:00
> From: <a01@a> & From: <a02@a> & From: <a03@a>
> To:  b01@b & To:  b02@b & To:  b03@b
> CC: c101@c & CC: c202@c & CC: c303@c
> Subject: Test-01 & Subject: Test-02 & Subject: Test-03

>(Message Filter rules)
> If from     contains N,  Add tag fN  (where N=1 to 3)
> If to       contains N,  Add tag tN  (where N=1 to 3)
> If cc       contains N,  Add tag cN  (where N=1 to 3)
> If to or cc contains 10, Add tag tc1
> If to or cc contains 20, Add tag tc2
> If to or cc contains 30, Add tag tc3
> If subject  contains N,  Add tag sN  (where N=1 to 3)

(Result after Run Filters on folder)
> X-Mozilla-Keys: f3 t1 t2 t3 c1 c2 c3 tc1 tc2 tc3 s3                                         
  Same result with Tb 2.0.0.14 and Tb trunk 2008/5/21 build.
  ==> From: a03@a, Subject: Test-03 (last header is used)
  ==> Message filter of "to/cc/to or cc,contains" works as expected
      even when multiple CC: headers exist, if local mail folder.

(Subject etc. in message list pane and message header display)
     Column header  Message List Pane       Header display
 (1) Subject        Test-03                 Test-01              <= Mismatch               
 (2) Sender         a03@a                   a01@a                <= Mismatch
 (3) Recipient      b01@b, b02@b, b03@b     b01@b, b02@b, b03@b    
 (4) Date           2003/01/03              2001/01/01           <= Mismatch
 (5) CC             (Not applicable)        c01@c, c02@c, c03@c
Note-1: When Tb trunk, column header of (2) is "From" instead of "Sender".
Note-2: Mismatch of (1), (2), (4) is already reported phenomenon to other bug.

(Q1) Is there any rule(which moves the mail to other folder) before the rule of "to or cc,contains"? If such rule exists, and move to the other folder fails by some reasons(e.g. compacting of the other folder is in progress), move silently fails, even though message filter log says "rule is fired & mail is moved".
Check message filer log first.

(Q2) IMAP?
When IMAP, header is obtained by following command first.
> UID fetch xxxx (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc
>   Subject Date Message-ID Priority X-Priority References Newsgroups
>   In-Reply-To Content-Type reply-to)])
So, when multiple CC: headers exist, filtering depends of behaviour of IMAP server on multiple CC: headers(I don't know what should be returned by IMAP server in this case).  

(Q3) If POP3(i.e. local mail folder), filtering result when mail download is possibly different from manual "Run filter on folder". Same result when manual "Run Filer on folder"?

(Q4) As my test result says, if multiple Date:,From:,Subject: headers exist, Tb currently uses last one(other bug). This may affect execution of other mail filter rules. Is there such filter rule in your message filter?

(Off-Topic)
> (its very difficult to get the header via cut/paste with Thunderbird)
Message headers can easily be copied via View/Message Source, although encoded non-ascii Subject: or encoded non-ascii name etc. is to be replaced by decoded text in message display pane.
Q1: no other rules, I have only one rule "to or cc contains clusterwr".
Q2:, yed we do have imap, we have here Oracle Beehive in use.
Q3: manual 'Run Now does not change anything.
Q4: I have approx 30 rules (!!) and the rule in question is 9th in my filter chain.

great, I have not spotted View/Msg Source. I used it now on another clusterwr message which was not moved to another folder:

[...
Received: from acsmt350.oracle.com (/141.146.46.1)
	by bhmail.oracle.com (Oracle Beehive Gateway v4.0)
	with ESMTP ; Fri, 23 May 2008 13:14:01 -0700
Return-Path: <RAJESH.DASARI@ORACLE.COM>
Received: from stadh28.us.oracle.com by acsmt351.oracle.com
	with ESMTP id 3679469851211573637; Fri, 23 May 2008 13:13:57 -0700
Subject: Any txt 
From:A.B@ORACLE.COM
To: 
CC: ANDREY.G@ORACLE.COM, CLUSTERWR_US@ORACLE.COM, G.B@ORACLE.COM, J.T@ORACLE.COM, R.D@ORACLE.COM, R.V@ORACLE.COM
CC: HELENA.K@ORACLE.COM
Date: Fri, 23 May 2008 13:13:57 -0700
...etc]

Of course, I made numerous experiments as if the filters are case sensitive... I have added numerous other filters as experiment, e.g. with entire string CLUSTERWR_US. All to no avail.

As I said, this filter problem was caused by one of the recent versions, I am not sure really which one. 
Actually I have probably misspoken here, I meant filters, not rules:

Q4: I have approx 30 filters (!!) and the filter in question is 9th in my filter
chain.

The filter itself has only one rule, as stated in Q1. No other filter uses this rule. The filters which I created by such non filtered messages as experiments are placed behind this filter with 'clusterwr'.
(In reply to comment #2)
> Q2:, yed we do have imap, we have here Oracle Beehive in use.

(Q5) What does filer log say for the unmoved/multiple CC: mail?
     "to or cc,contains,clusterwr" fired but not moved to target folder?
     Or no filter log for the multiple CC: mail?

(Q6) Where is the move target folder of the filter located?
     Other IMAP folder of same IMAP account? Folder of other IMAP account?
     Or local mail folder?(folder of "Local Folders" or POP3 account)?

(Q7) Do you set mail.imap.use_envelope_cmd=true?

Following is partial list of RFC 2822 section 3.6. RFC 2822 defines maximum number of CC: header as 1.
> http://www.faqs.org/rfcs/rfc2822.html
> 3.6. Field definitions
> Field           Min number      Max number
> from            1               1
> sender          0*              1
> to              0               1
> cc              0               1
> message-id      0*              1
> subject         0               1

Gmail IMAP returned multiple CC: headers for following request by mail.imap.use_envelope_cmd=false(default) when multiple CC: headers exist in a mail.
> UID fetch xxxx (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc ...
However, it is possibly different when ENVELOPE is requested on FETCH by 
mail.imap.use_envelope_cmd=true, because max number of CC: headers defined by RFC 2822 is one.

IMAP protocol level flow can be seen by NSPR log. See Bug 402793 Comment #1 for getting NSPR log.

(In reply to comment #2)
> Q4: I have approx 30 filters (!!) and the filter in question is 9th in 
my filter chain.

There are known bugs that causes mis-firing of filter rule, or that interfares filter action. For expmple,
 - Rule of "if cond-1 && cond-2" always fires even if cond-2==false.
   Reversed "if cond-2 && cond-1" works well. An exaple of cond-1 : Starred
 - Rule of "if sender" for customized sender(==Sender: header) checks
   From: header instead of requested Sender: header.
   Reversed "if cond-2 && cond-1" works well. An expmple of cond-1 : Starred
 - When both action of "copy to a folder" and action of "move to a folder"
   are executed for a mail, one of two actions can not be executed properly.
(Q8) Is there any other filter which possibly causes such problem thus this bug's problem?
I will followup on this next week in the office.

Regarding the violation of RFC 2822 rule of one CC: header, I will be happy to file a bug against Beehive.
Another RFC 2822 violation by your server.
RFC 2822 permits ZERO "To:" header line(max number of To: header==1).
However, RFC 2822 doesn't permit no data in To: header.
  According to my quick read of RFC 2822's BNC notation for To: header,
  at least "<>" or "@" should exist in To: header, if not mailing list format.
  Sorry but I haven't checked whether null local_part or null domain_part
  of local_part@domain_part format is permitted by RFC2822 or not.
  
Adding words for "null To: header & multiple CC: headers" in bug summary, for ease of search. Please remove them and modify bug summary, if real cause will be found.  
Summary: Message filter does not work properly → Message filter(to or cc,contains) does not work properly, if no parameter data in To: header and multiple CC: headers
Probably due to Bug 496046.
> Bug 496046 : Message headers are processed in reverse order
Status: UNCONFIRMED → NEW
Depends on: 496046
Ever confirmed: true
there is another bug in filters or search that talks to fields like To: that are null

one of them should be duped
Component: General → Filters
Product: Thunderbird → MailNews Core
QA Contact: general → filters
(In reply to Wayne Mery (:wsmwk) from comment #9)
> there is another bug in filters or search that talks to fields like To: that
> are null
> 
> one of them should be duped
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.