527216 - mail filter action "fetch From POP Server" fails when anti-virus helper option quarantine is enabled

Status: RESOLVED DUPLICATE of bug 548512

(MailNews Core :: Filters, defect)

Description

[Hannes]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6pre) Gecko/20091106 Shredder/3.0pre

It is not possible to retrieve emails that match the "fetch From POP Server" filter if the Anti Virus Option is enabled. Instead of this headers that don't match the filter are retrieved every time, a new matching one resides on the server.

Reproducible: Always

Steps to Reproduce:
1. create a pop account, that only fetches the header and leaves the messages on  the server
2. create a "fetch From POP Server" filter
3. send one email matching the filter and one email not matching the filter
4. try to get the matching one - repeat
5. send another matching one
6. try to get it - repeat
Actual Results:  
4.: The non matching header is retrieved for the first time. The matchimg one prefers to stay where it is. With trying again to get it the non matching header is fetched again.
6.: Nothing happens. With trying again to get it the non matching header is fetched again.
Repeating steps 5 and 6 will always non matching headers fetch again

Expected Results:  
non matching headers should only be fetched one time
emails with matching headers should be fetched 

This is the little brother of bug 315791 that can celebrate its 4th anniverary
when TB3RC1 will be released. It should be well known, that disabling the anti virus option will possibly cause lost of data of a complete email folder. I do not understand why a bug like this can survive for more than a whole TB generation!

Comment 1

[[:Aureliano Buendía]]

I cannot confirm here with your STR (steps to reproduce).

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6pre) Gecko/20091107 Lightning/1.0pre Shredder/3.0pre ID:20091107031309

Comment 2

[Hannes]

Attachment: Inbox - empty Filter-Folder

Comment 3

[Hannes]

(In reply to comment #1)
> Well, it seems that I've left out some steps which are important in this case.
Also I didn't notice that the complete non-matching emails are fetched and not only the headers as I described above.

I used two POP accounts to do that. I didn't try with only one until now.
The order of the following steps should not be changed!

Steps to Reproduce:
1. create a pop account(1), that only fetches the header and leaves the messages
on the server
2. disable the anti-virus option if necessary
3. send non-matching email from account 2 to account 1
4. create Filter "Fetch from POP Server" that filters the subject of the incoming emails
5. send matching email from 2 to 1
6. fetch both: header of non-matching email and complete email of matching    filter
result: both are now in the Inbox folder as expected

7. now create an extra folder for the filter as a subfolder of the Inbox  
8. expand the filter settings with new rule: "Move Message to MyFilterFolder"
9. set anti-virus to enabled
10. send matching email again from 2 to 1
11. try to get it - nothing happens
12. try again - the complete non-matching email is fetched and the matching one resides on the server
13. repeat steps 10 to 12 until it's getting boring 
result: see attachment and step 12


Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6pre) Gecko/20091107 Lightning/1.0pre Shredder/3.0pre

Comment 4

[Hannes]

Works using only one account with STR (steps to reproduce) as described in Comment #3 too.

Comment 5

[[:Aureliano Buendía]]

Confirmed here on 

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6pre) Gecko/20091107 Lightning/1.0pre Shredder/3.0pre ID:20091107031309

with steps from 1 to 6 (it is strange I know for you work fine): I can't see any message on my Inbox (1) also if message has filtered from TB: in log filter I see

Applied filter "TEST" to message from (:Aureliano Buendía) <[e-mail]> - Hello World at 08/11/2009 13.52.31 body fetched from POP3 server

Comment 6

[Hannes]

(In reply to comment #5)
 
> with steps from 1 to 6 (it is strange I know for you work fine): I can't see
> any message on my Inbox (1) also if message has filtered from TB: 

To avoid any misunderstanding: with steps 1 to 6 of comment #3 this would mean, that you cannot receive any email because you defined a filter? (Anti-Vir disabled?!)

>in log filter I see
> Applied filter "TEST" to message from (:Aureliano Buendía) <[e-mail]>
> - Hello World at 08/11/2009 13.52.31 body fetched from POP3 server

Yes, I confirm that. The log always lies, when things are getting worse.
In addition to this and step 11 of comment #3 the "statusbar-ticker" tells me:
"received 1 of 1 messages".

I tried to reproduce what you wrote about steps 1 to 6. To ensure that I have a clean platform, I uninstalled Shredder, deleted the profile and two folders in 
C:\Documents and Settings\username\.. and reinstalled Shredder. Until here the same procedure that I did before I posted comment #3.
But this time I created a profile with only one POP account and left out the installation of any extension. And now I get a new

result at step 12: nor header neither complete emails are fetched - repeating the steps 10 to 12 always have the same result
This is the same behaviour that you have at step 6, isn't it?

Comment 7

[[:Aureliano Buendía]]

(In reply to comment #6)
> (In reply to comment #5)
> 
> > with steps from 1 to 6 (it is strange I know for you work fine): I can't see
> > any message on my Inbox (1) also if message has filtered from TB: 
> 
> To avoid any misunderstanding: with steps 1 to 6 of comment #3 this would mean,
> that you cannot receive any email because you defined a filter? (Anti-Vir
> disabled?!)
No! I can confirm your result with steps 1-6 with "Allow antivirus clients to quarantine individual incoming messages" set to true.


> result at step 12: nor header neither complete emails are fetched - repeating
> the steps 10 to 12 always have the same result
> This is the same behaviour that you have at step 6, isn't it?
Yes it is.

Comment 8

[Hannes]

OK. I'am trying to reduce the amount of STR(steps to reproduce) at the moment to see, which of them are absolute necessary and which are not. 
This evening I succeeded again getting unwanted emails when wanted emails were expected (using two POP accounts again). Unfortunatly I didn't make screenshots this time. Please be patient for 1 or 2 days - then I will provide you with easier reproducable steps.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6pre) Gecko/20091109 Shredder/3.0pre

Comment 9

[Wayne Mery (:wsmwk)]

Hannes, how is this different from Bug 315791?   I don't see a difference.

Comment 10

[Hannes]

The process of debugging software in its beta stadium depends essentially on bug reports and  accuracy of these. The more a bug can be narrowed down the merrier are chances to fix it asap. Releasing the FOURTH beta version demonstrates impressively, how much Thunderbird‘s  development needs the community‘s participation during this process. SERIOUS treatment of bug reports has something to do with respect to the people who spent their time writing them.  

Aureliano, when I write “disable the anti-virus option“ , I just mean that and nothing else. I do not exactly know what you did, but you confirmed the title of this bug and not the result of my STR. This has nothing to do with accuracy. Would you have tried to repeat my STR step after step if I had entitled this bug with “MY mother makes the best spaghetti of the whole world!“? I bet you wouldn‘t have even tried and left the state UNCO or worse. ( )

Wayne, sure. Absolute identical. This one is a dup too: bug 387361. 
Before you write a comment to someone you don‘t know think twice. A good comment could have been: “Thank you Hannes for your report. High integrity of data is still a goal for us. Meanwhile we know everything about this kind of bugs and some people are working on it. See here and here and here.“ (a few links).

Sorry

Comment 11

[Wayne Mery (:wsmwk)]

I think you misunderstand. Yes, my comment is short and to the point. But if I didn't value your opinion, I wouldn't have asked, I would have just marked your bug a duplicate.  Yes, good bug reports are important, and your help is appreciated.  But I confess now to not understanding why you created it, now that you say this is a duplicate. If an existing bug covers your problem, the protocol in bugzilla is to improve on the existing bug, not file a new one.

You, Aureliano and Bienvenu might sort out what gets duped where - perhaps bug 387361 is the keeper. Perhaps David wanted both bugs. Or maybe 315791 just wasn't found at the time because a) it was in Prefs and b) it didn't have the word quarantine in the summary.  

Which is one thing I didn't understand when I first read your bug, why quarantine wasn't in the summary when it's one of the defining words of the preference.

Comment 12

[Hannes]

(In reply to comment #11)
> If an existing bug covers your problem, the protocol in bugzilla is to >improve on the existing bug, not file a new one.

After a test if the 'delete on server' action bug is still there I wanted to know if there might be another malfunction around filters. I performed the reported test and had to decide to open a new bug or just to expand the list of comments in 315791. In opposite to 315791 cmt. #9 emails never were moved to their folders. Due to former mentioned reasons I wanted to ensure to get most attention on this bug(s). Finally I decided to report a new one risking it to be marked as duplicate. Now in 315791 is a bit movement...  

> Which is one thing I didn't understand when I first read your bug, why
> quarantine wasn't in the summary when it's one of the defining words of the
> preference.

Might be dispensable. There is only one option.

Everything I hope should be clear now. 
I somewhere read that a whole bunch of code has to be rewritten to let the filters work as they should. If there should be no need to find out why or when their behavior is unstable we could do something else. What do you suggest?

Comment 14

[Wayne Mery (:wsmwk)]

actually, better duped to bug 548512