674954 - filtering an email is not possible when "=80" ... "=FF" is on the same line of the searched word in the content of the message

Status: UNCONFIRMED

(MailNews Core :: Filters, defect)

Description

[Damien]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0
Build ID: 20110615151330

Steps to reproduce:

I make a filter on my incoming messages, in order to move, in a dedicated directory, the messages whose body contains the searched word. 


Actual results:

the messages are well filtered until i receive a message where the sequence "=80" (without the ") appear on the same line of the searched word.
In that case the message is not filtered and is not moved in the directory and still in the incoming box.

The searched word is "einstein".
It appears in the body of the messages sent by my mantis server :
"
...
http://einstein/mantis/view.php?id=714 
...
"
But when the id reach number 800 it doesn't work :
for example the message that contains the following is not seen as containing the word "einstein" :
"
...
http://einstein/mantis/view.php?id=831 
...
"

I have mad other test and the pb is reproduce when one of the following sequences is found on the same line as the searched word :
"=80", "=81", ..., "=89", "=8a", ..., "=8f", "=8A", ..., "=8F", "=90", ..., "=FF"
in other way with the regular expression =[89A-Fa-f][0-9A-Fa-f]

but the pb does not appear when :
- the sequence is not on the same line of the searched word
- the search is made in the title and not in the body of the message
- the sequence is not the previously described (eg: %8F, $83, =8G, =G3, ...)


Expected results:

the search as not to depend of the presence or not of the described sequence on the line where the searched word is.

Comment 1

[Damien]

Attachment: an email where the word "stein" is not seen in the body

Comment 2

[Damien]

Attachment: an email where the word "stein" is seen in the body

Comment 3

[Damien]

Attachment: a screenshot of the filter (in french sorry)

Comment 4

[Ludovic Hirlimann [:Usul]]

Two more questions :

1) Does it happens in -safe-mode ?

2) Any errors in  Tools -> Error console ?

Comment 5

[Damien]

1) yes it happens also in -safe-mode

2) in safe mode, in the error console I've no error but only messages and warnings, like the followings, that do not concerne this pb (I think):
  - message : ***.com : server does not support RFC 5746, see CVE-2009-3555
  - warning : Avertissement : Propriété « box-sizing » inconnue.  Déclaration abandonnée.
Fichier Source : https://www.mozilla.org/fr/thunderbird/6.0/start/?uri=/thunderbird/start&locale=fr&version=6.0.1&os=WINNT&buildid=20110830105949
Ligne : 15

Comment 6

[Damien]

The problem can be reproduce on the new 7.0.1 version of Thunderbird.

Comment 7

[Wayne Mery (:wsmwk)]

aceman, does this depend on any mime issue?

Comment 8

[:aceman]

May be.

Comment 9

[WADA:World Anti-bad-Duping Agency]

Sounds problem of bug 667854. (See bug 667854 comment #11, please) 

> Bug summary: filtering an email is not possible when "=80" ... "=FF" 
>  is on the same line of the searched word in the content of the message

Seems plain text case(non base64, non quotd-printable case).

Comment 10

[WADA:World Anti-bad-Duping Agency]

I think "Body search" of Quick Search, Edit/Find/Search Messages, Saved Search folder(Virtual Folder, including Unified Folder), Message Filter, shares same code for searching, but problem may be in outside of search itself(for example, excess decoding as quoted-printable even though text/plain or text/html).
I'm not sure poblem in search and problem in filter is same issue.
Setting dependency to bug 667854 for ease of tracking and analysis.

Comment 11

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.

Comment 12

[Wayne Mery (:wsmwk)]

Danny, does this still reproduce?

Comment 13

[Dan Pernokis]

NO - The bug does not reproduce on my TB 91.2.1 under Windows 7/32 Pro (Aero).

The thing that struck me right away was the list running "80 .. .89 8a ... 8f 8g ...", where 8f fails and 8g works. "8f" is a valid HEX code, whereas "8g" is not, and I think the "=8f" (etc) was not parsing properly. But obviously the guys already suspected that (and probably fixed it).

Anyway, for testing, my filter checks: If body contains "stein", add a star. I sent six messages to myself, with and without "einstein" AND with or without "8f" or "8g" on the same or different lines.

If the body contains "einstein", there may or may not be a star, depending on presence and position of "8f" or "8g".
If the body does not contain "einstein" (contained "newton" instead) there should never be a star, and there never was.

(n) "Literal Body Content" [ Subject ] RESULT
(1) "Testing for einstein with =8f appearing" [code 8f appears - should NOT add star if bug present] STAR ADDED
(2) "Testing for einstein with =8g appearing" [code 8g appears - should ALWAYS add star] STAR ADDED
(3) "Testing for newton with =8f appearing" [code 8f appears but no stein - should NEVER add star] NO STAR
(4) "Testing for newton with =8g appearing" [code 8g appears but no stein - should NEVER add star] NO STAR
(5) "Testing for einstein with <newline> the =8f appearing on next line" [code 8f appears on next line - should add star] STAR ADDED
(6) "Testing for einstein with <newline> the =8g appearing on next line" [code 8g appears on next line - should add star] STAR ADDED

Case 1 in theory should succeed always, as "stein" appears ("einstein") If the bug is present, it should fail (NO STAR). But there is a STAR.
Case 2 should succeed always, as "stein" appears ("einstein") but contains "8g" instead of "8f". STAR ADDED -- correct.
Cases 3 & 4 should never succeed regardless of "8f" or "8g" (contains "newton" and no "stein", so match failed). NO STARS -- correct.
Cases 5 & 6 test for "stein" but have the "8f" or "8g" on separate lines, so they should always succeed, and they did. STARS ADDED -- correct.

So I would say this bug has been fixed (or at least is no longer evident).