368826 - filter doesn't find text, find does

Status: RESOLVED INCOMPLETE

(MailNews Core :: Filters, defect)

Description

[turningoff]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9
Build Identifier: 

I perhaps should break these up, but there are two sort of related topics about message filtering that I don't understand and wonder if there is a fix for.

(a) In one case I get an email with apparently just an image in it, and all the ones I have examined look the same whether I set view message body as plain text or html, I still see the gross picture, but one of my message filters that lets thru messages that have phrases of interest to me (usually one word in these cases) has dumped this **** in my inbox.

I am not a mail wizard, but I know when grubbing around that I have seen some images display as a bunch of text when I select show as plain text.  My guess is my key words are part of the image displayed this way, although I am perplexed as to why view as plain text still shows the image not the text.

Anyway, I am wondering if there is some way that the message filtering process can not search for words in what is an image - or am I misunderstanding what is letting this stuff through?

Unfortunately I didn't save an example.

(b) This is a semi-opposite case, in which a filter that should have caught an email containing spam words and thrown it in the trash didn't catch it.  I did do a find by hand on the message and verify that find thinks the words are there, so there aren't invisible characters messing up the filter search.

Is the encoding or something (I don't really know what I'm talking about here) in the body preventing the filter search from working?

The filter is if all of
full name
transfer
payment

are in the email body, trash it. Here's the email.





Content-Type: text/plain

Content-Transfer-Encoding: 8bit




Kindest Attention Dear Reader.



The SONATRACH PETROLEUM CORPORATION ( UK ), Urgently requires the

services of an earnest individual in the United States

for contract services on a part-time basis, .



NATURE OF SERVICES

(PAYMENT RECEIVING OFFICER) Such that He/She will be

receiving payments from our regular Clients/customers, So that we can

escape

some taxes which we normally pay each time we receive payments directly

from them via

international transfer into our account.



TERMS OF SALARY

If aprroved as our PAYMENT RECEIVING OFFICER,you are entitled to an

annual income of

$25,000USD and 15% of whatever amount you receive from our

Clients/Customers who are

making payments through you to us, and then 82% is for the company

while the remaining

3% goes for all expenses when sending the 82% to the company via

a means that will not disclose the money as being proceeds of exports

for which tax was evaded..



IMPORTANT NOTE

Note that you are not required to pay a dime for this job.

This service is part time job and will not disturb you or affect your

present job.



WHO WE ARE

Sonatrach Petroleum Corporation (SPC BVI) was created in 1989, in

accordance with the International Business Companies Ordinance of 1984.

It is a 100% owned subsidiary of ‘Sonatrach International Holding

Corporation’,

which is itself owned by the Sonatrach Group. Its Registered Office is

located

in The British Virgin Islands with a capital stock of 120 millions US

Dollars.



OUR ACTIVITIES

The main activities of SPC ( SONATRACH PETROLEUM CORPORATION) ,

are the trading of all Hydrocarbons, Liquids, Gas,Petroleum products,

and Shipping.



If you'r interested, kindly reply with the following:-



1.Full Names_________________________________________

2.House Address______________________________________

3.State:_____________________________________________

4.City:______________________________________________

5.Zipcode:____________________________________________

6.Phone Number________________________________________

7.Country_______________________________________________

8.Occupation_____________________________________________

9.Annual income__________________________________________



Announcer

Prof Reginald Williams

ADVERTISING TEAM

SONATRACH PETROLEUM CORPORATION

Reproducible: Always

Steps to Reproduce:
1.see above.
2.
3.

Comment 1

[David :Bienvenu]

We don't look for text in images, if by that you mean OCR'ing the image. But the mail could be hiding the text in some part of the message that's not displayed. The body search code just looks through all the text it can find in the message.

Re b), the filter body search uses the same code as the normal search/find code, I believe.

Comment 2

[turningoff]

No, I don't mean OCRing the image, I meant when I select view message body as plain text sometimes this humongeous body of text, presumably the image in some encoding, appears.  I thought maybe it was searching that.

So there can be non-displayed code in a message body, regardless of how I set view message body?  Is there any valid reason for that?  If not, it would be a good thing to add as an optional check in message filtering (non-displayed code, into the trash.)

As to b), then I don't understand how I can "find" the strings with find by hand, but the particular filter does not find them.  Does the encoding somehow mess this up?

Comment 3

[Mark]

When I do a search of my messages and search on the body, I often get false positives too.  The text is nowhere in the body at all.  But if I view the source and then search for the text, I find the search term in the base64 encoded section.  It sounds like the same thing is happening to you, but you find the text in an encoded image.

This is probably a variation on bug #268459, which looks like a fixed has been checked in and available in rc1. (Time for us to get on the beta train and check I guess.)  Might even be related to bug #249841 but hard to tell.

Comment 4

[John Kaufmann]

I agree with Mark that this is a variation on:
 bug #268459 - Search/filters for Message Body broken for quoted-printable bodies,
as well as:
 bug #252613 – Body contains search fails to find messages with "charset=ISO8859_1"

It is also more directly related not only to:
 bug #249841 - False positives from message bodies search ...
but also to:
 bug # 37031 - searching message body yields false positives ...
 bug #132340 - Local body search does not work if the body is encoded as Base64
 bug #328033 - "Search messages" returns results that do not contain the search string

The connection is that all of these search/filter bugs depend on an encoding problem, something that remains a fundamental design issue for Thunderbird. (Actually, #252613 is not really an example of the fundamental problem, just of the encoding being improperly specified.)  Two of those bugs (plus this one) are assigned to Scott MacGregor <[e-mail]>, so all three of those may be resolved at the same time.  The others are assigned to Jean-Francois Ducarroz <[e-mail]> (37031), Junkshik Shin <[e-mail]> (252613) and [for now] nobody (132340).  If they are all in communication, then presumably the others will be resolved at the same time, and one of the commonly observed (but little understood) problems of Thunderbird will be behind it.

Comment 5

[Wayne Mery (:wsmwk)]

turningoff, please attach a message that isn't detected by filter but is detected by find using version 2 or trunk build.

Comment 6

[Joshua Cranmer [:jcranmer]]

RESO INCO due to no response to wsmwk's question.