Closed Bug 162971 Opened 22 years ago Closed 19 years ago

Message body filters failing on certain messages.

Categories

(MailNews Core :: Filters, defect)

x86
Linux
defect
Not set
major

Tracking

(Not tracked)

RESOLVED EXPIRED

People

(Reporter: 6jju4k002, Assigned: naving)

Details

Attachments

(2 files)

Mozilla build: 2002071404

Description of the probem:
I have 2 sets of filters that don't function correctly.

1st is a filter on mail from the bugzilla system.
It moves this mail to a separate folder.
This filter is never applied.
  the condition line for this filter from the inbox's rules.dat is:
  condition="OR (from,begins with,bugzilla-daemon@mozilla.org)"
  
A search for mozilla mail with the same condition always succeeds.

2nd is 3 filters. First 2 filters filter mail from the debian_user &
debian_devel mailing lists. Third filter is catch all for any other
mailing list mails. These filters move the mails to appropriate folders.

The condition line for the debian-user list is:
   condition="AND (body,contains,debian-user-request@lists.debian.org) AND    
   (from,contains,Open source mailing lists)"
The condition line for the debian-devel list is:
   condition="AND (body,contains,debian-devel-request@lists.debian.org) AND       
   (from,contains,Open source mailing lists)"
The condition line for the catch all rule is:
   condition="OR (from,contains,Open source mailing lists)"

Most of the time the first 2 filters behave correctly. Sometimes though, these
filters fail and the catch-all rule is invoked.
A search with the same condition as debian-devel & debian-user fails.

Repeatability:
For the bugzilla mail is always repeatable.
For Debian mailing list mail is probably repeatable on average about once a day
over about a month(but I suspect that more spam is being rejected from lists, as
I haven't seen an occurance this week.)

Note on the debian mail:
I suspect that the filtering on body text may be a problem.
Much of the debian group spam is in the form of multi-part mime messages, http
messages, or messages with base64 or other encoding for body text. Sometimes the
text I'm searching for is in the body of the message, sometimes in a separate
mime element.
The "begins with" bugzilla-daemon filter works for me on the current commercial
branch (with win98, didn't yet set up on linux).  As for the body filters,
sounds like you may be running into a previously reported problem about body
searches and html tags.  Will look for the existing bug(s).
see bug 144036
Oh, your comment about win98 made me think:
I set OS as Linux on this bug. 
Infact I see this on Win98 & Linux.


The problems with the bugzilla filter were due to the fact that I receive
my mail through sneakemail. Either they added a quote character to their
From field, or the "begins with" filter of Mozilla wasn't working until
around the 2002071404 build.
The filter should now read:
condition="OR (from,begins with,\"\\"bugzilla-daemon@mozilla.org\")"
This is unlikley to work for people without a sheakemail account.

The debian mailing list issues still stand, with version 2002091908 of Mozilla.

What should I do with this bug?
Submit a new bug for the debian mailing list bugs? 
Mark this bug as invalid, or as duplicate of the above?
To clarify comment 4:

All my filters were moved to work on non-body fields, and to match the changes
to spancop. They now all work.

The filters were only buggy when filtering on body.
If anyone wans to imporve the performance of body filters, then using the body
condition on debian mailing lists may be a good test.

Questions in comment 4 about what the status of this bug should be still stand.
Rob: If you still have some of the emails that failed the body tests, maybe you
could examine them or attach them so we can see if this should be marked a dupe
of bug 144036 (or perhaps others) or is a new problem; otherwise I'd say we'd
best mark this WORKSFORME.
Just manually ran the old filters on the majority of old e-mails.
They work!
(Build is currently 2003013105)

Some mails remain, but they appear to be corrupted by a previous bug.
Will scan the mail file in an editor for the appropriate strings strings
that suggest the bug persists.

Will post again when I've completed my tests.

So, I think this works for me!
Further results:

Doesn't work for me.

A few e-mails still slip through the body filters.
Will post them to the appropriate bug, or if not post them here....

In which case, for a little while, the bug stays *unconfirmed*
Please ;-)
Some of my filters are failing consistently:

 name="CHI"
 enabled="yes"
 type="1"
 action="Move to folder"
 actionValue="mailbox://nobody@Local%20Folders/Grupos/CHI-ACM"
 condition="OR (to or cc,contains,CHI-WEB@ACM.ORG)"

After this I have tried many different conditions and moved the item up and down
without success.
I enclose the headers of a candidate message below.
Curiously enough, doing a search with the same condition identifies the messages
well.

Also, when I choose "Sender" the rules.dat contains:
    condition="OR (from,contains,chi-web@acm.org)"
where "Sender" is replaced by "from".
I added "Sender" as a custom header but it doesn't work too.

A sample non-filtered message, it didn't readt to "to or cc" and "sender"
conditions:

From - Sat Feb 08 01:05:58 2003
X-Mozilla-Status: 0011
X-Mozilla-Status2: 00000000
Return-Path: <owner-chi-web@ACM.ORG>
Received: from  rly-na05.mx.aol.com (rly-na05.mail.aol.com [172.18.151.234]) by
air-na01.mail.aol.com (v90_r2.5) with ESMTP id MAILINNA11-0207151943; Fri, 07
Feb 2003 15:19:43 1900
Received: from  mail.acm.org (mail.acm.org [199.222.69.4]) by
rly-na05.mx.aol.com (v90_r2.6) with ESMTP id MAILRELAYINNA510-0207151940; Fri,
07 Feb 2003 15:19:40 -0500
Received: from mail (mail.acm.org [199.222.69.4])
    by mail.acm.org (8.9.3/8.9.3) with ESMTP id OAA13336;
    Fri, 7 Feb 2003 14:56:02 -0500
Received: from ACM.ORG by ACM.ORG (LISTSERV-TCP/IP release 1.8d) with spool id
          610139 for CHI-WEB@ACM.ORG; Fri, 7 Feb 2003 14:52:20 -0500
Approved-By: singal2@percupingui.COM
Received: from alias2.acm.org (alias2.acm.org [199.222.69.92]) by mail.acm.org
          (8.9.3/8.9.3) with ESMTP id MAA13468 for <chi-web@listserv.acm.org>;
          Fri, 7 Feb 2003 12:04:49 -0500
Received: from andretti.dargundublis.co.uk ([213.86.174.18]) by alias2.acm.org
          (ACM Email Forwarding Service) with ESMTP id CRY73883 for
          <CHI-WEB@acm.org>; Fri, 07 Feb 2003 12:16:42 -0500
Received: from buddudungu.com (arcglade.demon.co.uk [::ffff:158.152.10.150]) (AUTH:
          PLAIN paola@dargundublis.co.uk,
          TLS: TLSv1/SSLv3,128bits,RC4-MD5) by andretti.dargundublis.co.uk with
          esmtp; Fri, 07 Feb 2003 17:17:40 +0000
User-Agent: Mozilla/5.0 (Windows; U; Win95; en-US; rv:1.2.1) Gecko/20021130
X-Accept-Language: en, en-us
MIME-Version: 1.0
References: <C1256CC6.0037F3C8.00@omext02.omgroup.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Message-ID:  <3E43EA31.2000506@buddudungu.com>
Date:         Fri, 7 Feb 2003 17:17:37 +0000
Reply-To: Paola Kathuria <paola@buddudungu.COM>
Sender: "ACM SIGCHI WWW Human Factors (Open Discussion)" <CHI-WEB@ACM.ORG>
From: Paola Kathuria <paola@buddudungu.COM>
Organization: Limitless Innovations
Subject:      Re: *ize vs. *ise  Was: US/UK English capitalisation
X-cc:         xper.langstrom@ACM.ORG
To: CHI-WEB@ACM.ORG
In-Reply-To:  <C1256CC6.0037F3C8.00@omext02.omgroup.com>
X-Mailer: Unknown (No Version)

xper.langstrom@OM.COM wrote: 
... snip ...
Surprise!
The notification mail for my recent Comment #9 got filtered.
The condition:
   condition="OR (\"Return-Path\",contains,owner-chi-web@ACM.ORG>"
catched it.
As the mail was originated in this Bugzilla, it's obvious that the text was
found in the message body, not in the "Return-Path: <nobody@mozilla.org>" header.
I think this might be of some help to isolate the problem.
In my testing I finally found a working filter.
The condition:
  condition="OR (\"Return-Path\",contains,owner-chi-web@ACM.ORG>"
moved the messages correctly. One of the filtered messages has the following
headers (saved as .eml):
.....................................................
From - Thu Feb 13 14:00:47 2003
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <owner-chi-web@ACM.ORG>
Received: from  rly-na04.mx.aol.com (rly-na04.mail.aol.com [172.18.151.233]) by
air-na02.mail.aol.com (v90_r2.5) with ESMTP id MAILINNA23-0213101724; Thu, 13
Feb 2003 10:17:24 1900
Received: from  mail.acm.org (mail.acm.org [199.222.69.4]) by
rly-na04.mx.aol.com (v90_r2.6) with ESMTP id MAILRELAYINNA45-0213101721; Thu, 13
Feb 2003 10:17:21 -0500
... snip ...
.....................................................

I purposedly chose a condition on one of the first headers. Other conditions
were testing headers found closer to the bottom of the list.
Maybe the filter program is quitting before checking all headers ...

Juan, I think your mail probems are not the same as mine.
Mine are (now) to do with search in message body filtering only.

I'd suggest looking around for bugs that involve the message fields you're
having problems with, and if you can't find any, open a bug for that problem.
This is the first testcase for this bug.
See later comments for details.
The second test case is too large to upload as an attachment, (525.7KB)
so, will supply it to interested parties on demand.

Both test cases are mozilla mail files.
They contain messages that couldn't be filtered with the
following filter rules:

name="Debian User"
enabled="yes"
type="1"
action="Move to folder"
actionValue="mailbox://XXXXXX/Inbox/Debian%20mail%20that%20matches%20user"
condition="AND (body,contains,debian-user-request@lists.debian.org)"
name="Debian devel"
enabled="yes"
type="1"
action="Move to folder"
actionValue="mailbox://XXXXXXX/Inbox/Debian%20mail%20that%20matches%20devlel"
condition="AND (body,contains,debian-devel-request@lists.debian.org)"

Thse filters were last attmpted on these files with the 2003041008 mozilla nightly.

A succesful filter will move all the messages from these mail files.
Currently these filters move none of the messages.

From examining the messages in the attachments it can be seen that while
some of the messages are likely to fail because of known bugs, other filter
failures do not correspond to any knwon bugs.
Compressed the test case to allow bugzilla to accept it.
Summary: Some Filters not being applied, some messages not being filtered → Message body filters failing on certain messages.
Rob:

In my comment #10 I tell that I noticed that a filter supposed to act on a 
header reacted when that header was in the body of a message about this bug 
sent by bugzilla-daemon.
The filter seems not to be aware of what part of the message it's processing.

You wrote:

> Juan, I think your mail problems are not the same as mine.
> Mine are (now) to do with search in message body filtering only

but maybe it's all the same and it can be fixed all at once.

Also, I don't insist much because I'm not in Mozilla but Netscape 7. But if you 
want I can send you several messages to be used in testing.

Saludos
Juan Lanus
Juan, 

Hmm, now you put it that way, it is an interesting bug.

The real reason you should report this somewhere else is that this bug is badly
defined (by me) and therefore largely ignored by the QA and developer it's
assigned to.
I think you'd get a better response from the mozilla people with a new bug.

Also, the Mozila developers prefer to have bugs like these separated so that
if the bugs don't have the same cause, one can be fixed, and the other left 
open.

Product: MailNews → Core
This is an automated message, with ID "auto-resolve01".

This bug has had no comments for a long time. Statistically, we have found that
bug reports that have not been confirmed by a second user after three months are
highly unlikely to be the source of a fix to the code.

While your input is very important to us, our resources are limited and so we
are asking for your help in focussing our efforts. If you can still reproduce
this problem in the latest version of the product (see below for how to obtain a
copy) or, for feature requests, if it's not present in the latest version and
you still believe we should implement it, please visit the URL of this bug
(given at the top of this mail) and add a comment to that effect, giving more
reproduction information if you have it.

If it is not a problem any longer, you need take no action. If this bug is not
changed in any way in the next two weeks, it will be automatically resolved.
Thank you for your help in this matter.

The latest beta releases can be obtained from:
Firefox:     http://www.mozilla.org/projects/firefox/
Thunderbird: http://www.mozilla.org/products/thunderbird/releases/1.5beta1.html
Seamonkey:   http://www.mozilla.org/projects/seamonkey/
This bug has been automatically resolved after a period of inactivity (see above
comment). If anyone thinks this is incorrect, they should feel free to reopen it.
Status: UNCONFIRMED → RESOLVED
Closed: 19 years ago
Resolution: --- → EXPIRED
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: